oylz / mmdnn

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK and PyTorch.

MMdnn

A comprehensive, cross-framework solution to convert, visualize and diagnosis deep neural network models. The "MM" in MMdnn stands for model management and "dnn" is an acronym for deep neural network.

Basically it converts many DNN models that trained by one framework into others. The major features include:

• Model File Converter Converting DNN models between many frameworks
• Model Code Snippet Generator Generating training or inference code snippet for any frameworks
• Model Visualization Visualizing dnn network structure and parameters for any framework
• Model compatibility testing (On-going)

Features

Model Conversion

Across the industry and academia, there are a number of existing frameworks available for developers and researchers to design a model, where each framework has its own network structure definition and saving model format. The gaps between frameworks impede the inter-operation of the models.

We provide a model converter to help developers convert models between frameworks, through an intermediate format.

The intermediate representation will store the network structures as a protobuf binary and pre-trained weights as NumPy native format.

[Note] Currently the IR weights data is in NHWC (channel last) format.

Support frameworks

• Caffe (Source only)
• Keras
• MXNet
• Tensorflow (Experimental)
• Microsoft Cognitive Toolkit (CNTK) (Destination only)

Tested models

The model conversion between current supported frameworks is tested on some ImageNet models.

Models	Caffe	Keras	Tensorflow	CNTK	MXNet	PyTorch
Inception V1	√	√	√	√	√
Inception V3	×	√	√	√	√
ResNet V1 50	×	√	√	o	√	√
ResNet V2 152	×	√	√	√	√
VGG 19	√	√	√	√	√	√
MobileNet_v1	×	√	√	×	×
Xception	×	√	√	×	×

On-going frameworks

• Caffe2
• CoreML
• PyTorch

Usage

We will use the conversion from Keras "inception_v3" model to CNTK as an example.

Install keras and tensorflow in case

$ pip install keras
$ pip install tensorflow
$ pip install 

1. The example will download the pre-trained models at first, then use a simple model extractor for Keras applications, you can refer it to extract your Keras model structure and weights.

$ python -m mmdnn.conversion.examples.keras.extract_model -n inception_v3

Using TensorFlow backend.
Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.5/inception_v3_weights_tf_dim_ordering_tf_kernels.h5
96075776/96112376 [============================>.] - ETA: 0s
.
.
.
Network structure is saved as [imagenet_inception_v3.json].
Network weights are saved as [imagenet_inception_v3.h5].

The structure file imagenet_inception_v3.json and weights file imagenet_inception_v3.h5 are downloaded to current working directory.

2. Convert the pre-trained model files to intermediate representation

$ python -m mmdnn.conversion._script.convertToIR -f keras -d converted -n imagenet_inception_v3.json -w imagenet_inception_v3.h5

Using TensorFlow backend.
.
.
.
Network file [imagenet_inception_v3.json] is loaded successfully.
IR network structure is saved as [converted.json].
IR network structure is saved as [converted.pb].
IR weights are saved as [converted.npy].

The Command will take imagenet_inception_v3.json as network structure description file, imagenet_inception_v3.h5 as pre-trained weights, and you will get the intermediate representation files converted.json for visualization, converted.proto and converted.npy for next steps.

3. Convert the IR files to CNTK models

$ python -m mmdnn.conversion._script.IRToCode -f cntk -d converted_cntk.py -n converted.pb -w converted.npy

Parse file [converted.pb] with binary format successfully.
Target network code snippet is saved as [converted_cntk.py].

And you will get a file name converted_cntk.py, which contains the original CNTK codes to build the Inception V3 network.

With the three steps, you have already converted the pre-trained Keras Inception_v3 models to CNTK network file converted_cntk.py and weight file converted.npy. You can use these two files to fine-tune training or inference.

4. Test the converted model

$ python -m mmdnn.conversion.examples.cntk.imagenet_test -p inception_v3 -s keras -n converted_cntk -w converted.npy
.
.
.
[(386, 0.94166422), (101, 0.029935161), (385, 0.0025184231), (340, 0.0001713269), (684, 0.00014733501)]
Test model [inception_v3] from [keras] passed.

The converted model has been tested.

5. Dump the original CNTK model

$ python -m mmdnn.conversion.examples.cntk.imagenet_test -n converted_cntk -w converted.npy --dump cntk_inception_v3.dnn
.
.
.
CNTK model file is saved as [cntk_inception_v3.dnn], generated by [converted_cntk.py] and [converted.npy].

The file cntk_inception_v3.dnn can be loaded by CNTK directly.

Model Visualization

Some tools are provided to visualize the model network structure and settings. Take [Keras "inception_v3" model] as an example again.

1. Download the pre-trained models

python -m mmdnn.conversion.examples.keras.extract_model -n inception_v3

2. Convert the pre-trained model files into intermediate representation

python3 -m mmdnn.conversion._script.convertToIR -f keras -d converted -n imagenet_inception_v3.json -w imagenet_inception_v3.h5

3. Open the simple model visualizater [visualization/index.html] and choose file converted.json

Installation

You can use pip to install MMdnn.

Contributing

We are working on other frameworks conversion and visualization, such as PyTorch and CoreML. And more RNN related operators are investigating. Any contributions and suggestions are welcome!

Thanks to Saumitro Dasgupta, the initial codes of caffe-to-tensorflow are references to his project caffe-tensorflow.

Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.microsoft.com.

When you submit a pull request, a CLA-bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., label, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.