The tool was built for AI hardware/software architectures to quickly understand and analyze the overall and layer-wise structure of neural network models.
The types of results will be generated by the tool:
1) tables of parameters & statistics of network structure in excel format
2) graphs of model structures in pdf format
It is recommended to install the tool in a virtual environment, explained in the manual.
1.1. clone the repository to the local drive.
1.2. The tool works with :
Python 3.6+, Tensorflow 2.1 +, Pytorch 1.5+
1.3. Install the following Python modules:
pandas, numpy, matplotlib, openpyxl, scikit-learn, scikit-image, graphviz, python-graphviz, pydot
There are two versions of tools integrated in the package, one for ML framework pytorch: torch2table; the other is for Tensorflow via keras: keras2table.
- type the command as the following format to get the results
python torch2table.py -n resnet50
- six optional arguments are:
-n: the name of the neural network model. Tested models are listed below. Please note that the name is case-sensitive
-b: batch-size of the input data, default at 1
-e: element size in byte, for example, float32 = 4, int8 = 1, etc., default at 1
-c: channel size of the input data tensor, default at 3 (for RGB pictures)
-H: height size of the input data tensor, default at 224
-W: width size of the input data tensor, default at 224
--backward: boolean indicator of whether or not to include backward ops statistics, default at false
- The results are exported as the excel tables in ''/output/torch/''.
- torchvision:
- base models
alexnet, vgg11, vgg13, vgg16, vgg19, vgg11_bn, vgg_13 bn, vgg16_bn, vgg19_bn, resnet18, resnet34, resnet50, resnet101, resnet152, squeezenet1_0, squeezenet1_1, densenet121, densenet_169, densenet_201, densenet_161, googlenet, inception_v3, shufflenet_v2_x'n'_'n', mobilenet_v2, resnext50_32x4d, resnext101_32x8d, wide_resnet50_2, wide_resnet101_2,mnasnet'n'_'n'
- detection model
maskrcnn
- Recomendation:
dlrm
- RNN network:
lstm, gru, gnmt
-
one stage detection:
ssd_mobilenet, ssd_r34 -
Others Models can also be imported, as long as a input tensor is provided with the model. Please refer to Section 4 for details.
NOTE:
-
gnmtmodel requiresseq2seq, which may ask you to downgrade yourtensorflowfrom 2.1+ back to 1.13. Remember to upgrade it back to 2.1+ for other models when needed. -
Some models, such as
densenet, will take significantly longer to render network graph because of its complexity. Based on our test (with CPU instead of GPU),densenet121takes about several minutes to finish, whiledensenet161/densenet169would require more than 2 hours to generate corresponding PDF file. -
inception v3model will pop size inconsistency error when default values ofHandWare used, because 224 is too small to have valid padding/kernel size after certain layers of processing, one can fix it by switching to a bigger initial value. E.g.python torch2table.py -n inception_v3 -H 299 -W 299 -
The backward computation only covers CNN now and RNN is yet to include. We encourage interested readers to try adding them or work with us together in the future.
- type the command as the following format to get the results
python keras2table.py -n ResNet50
- three optional arguments are:
-n: the name of the neural network model. Tested models are listed below. Please note that the name is case-sensitive
-b: batch-size of the input data, default at 1
-e: element size in byte, for example, float32 = 4, int8 = 1, etc., default at 1
-c: channel size of the input data tensor, default at 3 (for RGB pictures)
-H: height size of the input data tensor, default at 250
-W: width size of the input data tensor, default at 250
--backward: boolean indicator of whether or not to include backward ops statistics, default at false
- The results are exported as the excel tables in ''/output/tf/''
- keras pretrianed models:
'DenseNet121', 'DenseNet169', 'DenseNet201', 'InceptionResNetV2', 'InceptionV3','MobileNet', 'MobileNetV2', 'NASNetLarge', 'NASNetMobile','ResNet101', 'ResNet101V2', 'ResNet152', 'ResNet152V2', 'ResNet50', 'ResNet50V2', 'VGG16', 'VGG19', 'Xception'
-
Reomendeation:
din -
EfficientNet:
EfficientNetB0 ~ EfficientNetB7 -
NLP:
bert
- keras-bert module should be installed for analysis,
pip3 install keras-bert
-
For DIN model, please clone the packages from tensorflow/models to a local folder, and add the folder into PYTHONPATH
-
Same as PyTorch version, the backward computation only covers CNN now and RNN is yet to include. We encourage interested readers to try adding them or work with us together in the future.
There are two types of output files in the /outputs/torch/ ( or /tf) folder: tables in xlsx and graph in pdf. Both are named as the input name of neural networks.
-
Total counts of memory and computation costs.
-
Note that 1M = 1024 x 1024 for data size and 1G = 1E9 for ops counts
- The results are demonstrated at one nn layer per row. The meanings of columns as below:
Layer:
TF Keras: Layer names & Types
Pytorch: layer names in multi-levels. Pytorch models are organized in a nested style. For example，a model may have several sequential/ sub-modules, and each module also have several nn layers. The first columns in the table demonstrate the hierarchical structures: the layer names in the first columns are at the top level of the model structure, the layer names in second column are at the second level of the model, etc. , as shown below,
layer-l0 | layer_l1 | layer-l1
├─Sequential: 1-5 Sequential:1-5 | |
| └─Bottleneck: 2-1 | Bottleneck:2-1 |
| | └─Conv2d: 3-1 ==> | | Conv2d: 3-1
| | └─BatchNorm2d: 3-2 | | BatchNorm2d: 3-2
| | └─ReLU: 3-3 | | ReLU: 3-3
Pytoch Model Hierarchy pytorch table
Input/Output tensors:
- TF Keras version, the shape of the tensor is formed as (height, width, channel)
- Pytorch version, the shape of the tensor is formed in (channel, height, width)
Kernel Tensors:
k1,k2: kernel size H&W for conv & pooling, labeld in Height, Width
s1,s2: stride H&W of rolling windows, labeld in X,Y
p1,p2: padding size, values are calculated based on centric padding, labeld in X,Y
Memory Costs:
- Size of input tensors
- Size of output tensors
- Size of model parameters，note that the values depends on Byte per Elements(BPE) of the models, default BPE is 4 (fp32)
Computation Costs: (Applies to both forward and backward ops)
- GEMM: Number of Matrix multi-adds
- ElemWise: Number of Element-wise Ops
- Activation: Number of activations(for transcendental Functions). Relu is also counted as an activation operation for convenience.
misc:
additional input/output tensors of the nn layer
Color bars of extreme cells
The cells/layers with the maximum cost are marked as:
- The maximum output tensor: Red,
- The maximum weight tensor: Pink
- The maximum forward matrix multi-add costs : Green.
- The maximum backward (if applicable) matrix multi-add costs : Yellow.
A graph to visualize the model structure will be also generated with the same name as the table file.
One can simply leverage the API in 'newmodel.py' in the root to analyze any customized models, as long as the models are built-up and ready for test.
There are two major steps for a new model: 1 set the model info ; 2 execute the main parser codes.
To parse the nn model, all required configures should be provided in function 'pymodel()' in 'newmodel.py'.
- model_path: the absolute path which contains the 'model_file'
- load model:
from 'model_file' import 'your_model'
model = your_model(args) # args: parameters of your model
- define inputs of the model: x = torch.rand(1,3, 300, 300) # tensor in (BCHW) format
You may define the input tensors for the customized model. Multiple inputs are formated as a list of tensors, as shown in the demo.
type the command as the following format to get the results:
python torch2table.py -n newmodel --model your_model
The optional argument --model is to get the model name. If the no model name provided, the results of the model will be saved as ' newmodel.xlsx' in the outputs folder.
The operations for Keras model are similar to pytorch model:
1 set the model configs. in 'tfmodel()' in 'newmodel.py';
2 execute command
python keras2table.py -n newmodel --model your_model
One can further analyze the various configures of a NN models by changing default settings in the codes. Two examples are shown below.
Note that the functions ( getmodel() & modelLst() ) mentioned below also demonstrate the ways to add a neural network model. Additional neural network models can be added using the similar approaches.
-
Bert models with different settings
The setting of bert model can be obtained by changing the optional arguments of the 'get_model' function at line 320 in //utils//tftools//getmodel(), please refer to the keras-bert for detailed settings for the bert model
-
inference model
To get bert inference model, two revisions are required:
-
change training = Flase at line 310 //utils//tftools//getmodel()
-
revise
return inputs, transformed->return inputs, transformed,modelat the orignal scirpts at line164 at \\lib\\site-packages\\keras_bert\\bert.py. Note that the operation will change the original codes of keras-bert
-
- Different settings of dlrm model
The setting of dlrm model can be adjusted by changing values of variables at line62-72 in in //utils//pytools//modellst(). Please refer to DLRM for the details of these variables.
Please note that special network structures and customized operators may not be supported by the tool. Feel free to join us to extend the functions of the tool.
- Codes for Pytorch model estimation were revised based on torch-summary @TylerYep
- Keras version EfficientNet is originated from EfficientNet @qubvel
- DLRM is originated from DLRM by facebook
- ML Perf models from ML perf @mlperf
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent