adlik / adlik Goto Github PK

When I compile the model to OpenVINO type, the error is as follows:
AttributeError: module 'tensorflow' has no attribute 'NodeDef'

You can build Adlik inside the Docker image.

Image is not runtime, so you can't compile Adlik inside image. There are two ways to use the image:

1. Use the image as a base image.
2. Run container of the image.

In the readme.md

Build serving with OpenVINO runtime
Install intel-openvino-ie-rt-core package from OpenVINO.

If we install intel-openvino-runtime-ubuntu18-2020.1.023 and intel-openvino-dev-ubuntu18-2020.1.023 according to the OpenVINO, we cannot successfully build the serving.

Support Benchmark test :

1. Test performance of different runtime framework, if Adlik supports
2. Test Adlik inference scheduler performance

When i compile the onnx model to OpenVINO or TensorRT model , the error is as follows：
TypeError: expected str, bytes or os.PathLike object, not NoneType {'status': 'failure', 'error_msg': 'expected str, bytes or os.PathLike object, not NoneType'}
But there is an onnx model in the directory of input_model.

When I use the docker images which built by ci/docker/build.sh file to compile the openvino model, the error message is as follows:

{'status': 'failure', 'error_msg': 'mo.py does not exist, path: /opt/intel/openvino_2019.3.344/deployment_tools/model_optimizer/mo.py'}

User usually get model meta which described in "model.pbtxt" to construct input before do prediction. But prediction will fail if information in "model.pbtxt" and model representation not consistent.

When i use the tensorrt 7 to compile the onnx model, raise parser error.

Related: #152.

Add markdownlint to CI for checking Markdown files.

Adlik does compression optimization for the model, but ONNX Runtime, Baidu's PaddleSlim and others all have similar functions, so what's the difference between them?

Add benchmark test result with tf lite runtime.
Model is Resnet50.

What is the meaning of “Adlik”? How to pronounce it? Add these information to README.md file.

Add the following CI checks:

• Build clients on macOS
• Build serving on macOS
  • Build serving with OpenVINO runtime on macOS
  • Build serving with TensorFlow CPU runtime on macOS
  • Build serving with TensorFlow GPU runtime on macOS
  • Build serving with TensorRT runtime on macOS

Support activate a specific model ( or version) via grpc/http interface

To complete model compiler scenario.

There is a bug in line 326 of model_loader.py in model_compiler

 if len(self.input_formats) < len(self.input_names):
      self.input_formats.extend([None for _ in range(len(self.input_formats), len(self.input_formats))])

Support update model (but not activate) via grpc/http interface

The introduction part of adlik adds a little more description of application scenarios and advantages over other platforms

The second line should start with “Model Compiler”.

The function “exit” does not belong to the list of async-signal-safe functions.
I guess that a different program design will be needed for your function “handler”.

ML runtime support http req.

Monitor the server's status which includes power on status and running status

When the model compiler creates config.pbtxt, if the Number dimension of the input and output shapes is not None but a specific value, the created config.pbtxt cannot be inferred.

An extra null pointer check is not needed in a function like the destructor for the class “Instance”.

Run tensorRT model failed because some codes about cuda not be compiled into binary.

Different runtimes have their own definition for model config

1.'graph conversoin' should be 'graph conversion'.
2.'OP combinaton' should be 'OP combination'.

Considering two scenario:

• Use QEMU ARM64 emulator.

1. Provide cross-compile tool-chain for ARM-64.
2. Compile ServingLite framework and TfLite runtime, and run inference test with Tflite Model ( Resnet50).
3. Considering several optimizing methods for TfLite Model, the specification of performance tests should be clear, and cover as many scenes as possible.

• Support real device such as Raspberry Pi or Nvidia Jetson Nano.

1. Provide specific cross-compile tool-chain.
2. Compile ServingLite framework and TfLite runtime, and run TfLite Model in device successfully.

Model compiler should support compiling framework for the Deep Learning model running in FPGA.

Adlik should support the framework to optimize deep-learned model, such as tensorflow checkpoint.
The optimizing techniques include:

1. Model Quantilization.
2. Model Pruning.

The "ModelOperateImpl::activateModel" function is too long and into chaos

After installing all of Adlik's packages and the build is complete, there should be a sample test to see if Adlik was successfully installed, similar to the "Hello world!" test of other software.

Currently, there are some bad designs in the model compiler. We should come up with a new design.

Support query model information include version, path, whether activated and so on

TensorFlow runtime doesn't enable any batch scheduler and can't batch multiple requests into one.

Support redirect log to file

For a model which has 3 versions: 1, 2 and 3, when adlik_serving starts it doesn't load version 3 but load version 2.

"" Build serving with OpenVINO runtime
Install intel-openvino-ie-rt-core package from OpenVINO.""

Is a installation process or a building process?

ServingLite runtime framework should support Deep Learning model running in FPGA.

1. Execute instruction pipeline by model compiler.
2. Divide ops running in CPU and FPGA.
3. Run ops in CPU
4. Call FPGA interface when running ops in FPGA.