skku-eslab / ant-model-db

ANT framework's model database that provides DNN models for the various range of IoT devices

ANT Model DB

This project includes model database, model profilers, and model dashboard used in ANT framework.

The model database provides DNN models for various range of IoT devices.

Model/Kernel Database

Model database records compressed DNN models.

Kernel database tunes kernel codes of those compressed models and collects them.

• Usage

python modeldb_sender.py --table <model_db_table_name> --path <directory_to_record> [--save_as_csv]

Model Profiler

Model profiler can measure latency using programs in latency-predictor, which implemented based on TVM.

References

This project is implemented based on TVM

• TVM https://github.com/apache/incubator-tvm

Prerequisites

• llvm Download Page
• CUDNN Download Page
• CUDA

Model Dashboard

Model dashboard is a web page that shows available compressed model in model database and kernel code in kernel database. This dashboard web server is implemented based on Flask.

• Usage

python dashboard-run.py 

License

ANT-Model-DB is open source software under the Apache 2.0 license.

ant-model-db's Issues

llvm, CUDNN, CUDA version

Does the version of prerequisites such as llvm and cudnn matter?

security measures

Are there any security measures in place for the model and kernel databases, especially considering the deployment of models on IoT devices?

Model DB support

In order to help users to choose the appropriate model, it is necessary to provide profiled information on the models.

So we decided to manage the model database along with the latency profiler.

The model DB needs to be implemented in this repository.

Usage and Setup

I'm new to programming and this field. Could you provide detailed instructions on how to set up and run the dashboard using dashboard-run.py?

Kernel Database

How does the kernel database tune the kernel codes of the compressed models?

Require web-based model dashboard

We've already developed the model database,
but the web-based model dashboard have not been uploaded to this repository.

To provide the users easy model selection UI, we need web-based model dashboard.

Guide to launch model dashboard

Hello,

Model dashboard seems to be uploaded.
However, since there is no guide to launch the model dashboard, I cannot use it.
Could you upload the guide to the README.md?

Basic Tutorials

Are there any resources, like documentation or tutorials, available for new users to understand and use the ANT Model DB effectively?

Broken pipe

When I run test.py with model=nvidia using RPC to work on odroid-xu4, it generates error as following.

TVMError: Socket SockChannel::Send Error:Broken pipe

It seems this error have something to do with parameters but I cannot find why.
What would be wrong?

Cannot find config

When I run a model including a 1x1 convolution layer on cuda, it warns me of possible performance degradation.

How can I make this layer find proper configuration?

Model types

What types of DNN models are included in the model database, and how are they optimized for IoT devices?

'XGBoostCostModel' object has no object 'pool'

Hi,
I installed requirements and now try to run test codes.

When I run test.py, this error message occurs and running fails.

AttributeError: 'XGBoostCostModel' object has no attribute 'pool'

My Environment is ubuntu18.04 and NVIDIA GPU.

Can you tell me how to fix this?

How the predictor predicts latency

Hi,
I want to ask how the predictor will predict the latency of a DNN model.
By machine learning or some other method?
And how many and what kinds of factors do you think counts to predict latency now?

DNN models update

How often is the model database updated with new DNN models?

Add License file

This repository has no license file.

If you extend TVM to implement Latency Predictor, you must specify your license referring to TVM's license.

TVM seems to use "Apache 2.0" License.

I think that Apache 2.0 license is proper for this repository.

Kernel DB support

To implement a latency predictor based on the regression model, we have to consider many cases.

So, it seems a need for kernel DB.

Can you confirm what models are available?

Can you confirm what models are available?

Add dependencies

There are many dependencies to use in several environments like

BLAS libraries, CUDNN, etc.

We have to specify how to install dependencies in README.

Separate TVM code from model profiler

As now, ANT model DB repository includes the source code of TVM.
As I think, the reason may be that the model profiler (latency predictor) has very strong dependency on TVM.

However, it is not easy to identify what is the source code of the model profiler.
For the maintenance in the long term, we need to separate the TVM code from this repository.
And then, we need to make TVM as a git submodule.

Readable Documentation

As model database and latency predictor are implemented, someone can want to use these modules.

However, it seems that there is no readable documentation in this repository.
I think that any readable documentation in the code repository or Github Wiki is required.

I would like to contribute to the project

How can someone contribute to this project? Are there specific guidelines or areas where help is needed?

Sparse kernel support for TVM model converter

Performing a sparse kernel is essential to measure the latency of various compressed models.

Currently, TVM supports model convert only using dense kernel.

It is necessary to find out the sparsity of the model and convert it using sparse kernel accordingly.