pinto0309 / simple-onnx-processing-tools

A set of simple tools for splitting, merging, OP deletion, size compression, rewriting attributes and constants, OP generation, change opset, change to the specified input order, addition of OP, RGB to BGR conversion, change batch size, batch rename of OP, and JSON convertion for ONNX models.

simple-onnx-processing-tools

A set of simple tools for splitting, merging, OP deletion, size compression, rewriting attributes and constants, OP generation, change opset, change to the specified input order, addition of OP, RGB to BGR conversion, change batch size, batch rename of OP, and JSON convertion for ONNX models.

1. Tools

HostPC

# (1) Minimum configuration installation with no dependent packages installed
$ pip install -U simple-onnx-processing-tools \
&& pip install -U onnx \
&& python3 -m pip install -U onnx_graphsurgeon --index-url https://pypi.ngc.nvidia.com

or

# (2) When installing all dependent packages such as onnx-simplifier, onnxruntime, numpy, etc...
$ pip install -U simple-onnx-processing-tools[full] \
&& pip install -U onnx \
&& python3 -m pip install -U onnx_graphsurgeon --index-url https://pypi.ngc.nvidia.com

Docker

$ docker run --rm -it \
-v `pwd`:/workdir \
-w /workdir \
ghcr.io/pinto0309/simple-onnx-processing-tools:1.1.30

No.	Tool Name	Tags	Summary
1	snc4onnx		Simple tool to combine(merge) onnx models. Simple Network Combine Tool for ONNX.
2	sne4onnx		A very simple tool for situations where optimization with onnx-simplifier would exceed the Protocol Buffers upper file size limit of 2GB, or simply to separate onnx files to any size you want. Simple Network Extraction for ONNX.
3	snd4onnx		Simple node deletion tool for onnx. Simple Node Deletion for ONNX.
4	scs4onnx		A very simple tool that compresses the overall size of the ONNX model by aggregating duplicate constant values as much as possible. Simple Constant value Shrink for ONNX.
5	sog4onnx		Simple ONNX operation generator. Simple Operation Generator for ONNX.
6	sam4onnx		A very simple tool to rewrite parameters such as attributes and constants for OPs in ONNX models. Simple Attribute and Constant Modifier for ONNX.
7	soc4onnx		A very simple tool that forces a change in the opset of an ONNX graph. Simple Opset Changer for ONNX.
8	scc4onnx		Very simple NCHW and NHWC conversion tool for ONNX. Change to the specified input order for each and every input OP. Also, change the channel order of RGB and BGR. Simple Channel Converter for ONNX.
9	sna4onnx		Simple node addition tool for onnx. Simple Node Addition for ONNX.
10	sbi4onnx		A very simple script that only initializes the batch size of ONNX. Simple Batchsize Initialization for ONNX.
11	sor4onnx		Simple OP Renamer for ONNX.
12	soa4onnx		Simple model Output OP Additional tools for ONNX.
13	sod4onnx		Simple model Output OP Deletion tools for ONNX.
14	ssi4onnx		Simple Shape Inference tool for ONNX.
15	sit4onnx		Tools for simple inference testing using TensorRT, CUDA and OpenVINO CPU/GPU and CPU providers. Simple Inference Test for ONNX.
16	onnx2json		Exports the ONNX file to a JSON file.
17	json2onnx		Converts a JSON file to an ONNX file.
18	sed4onnx		Simple ONNX constant encoder/decoder. Since the constant values in the JSON files generated by onnx2json are Base64-encoded values, ASCII <-> Base64 conversion is required when rewriting JSON constant values.
19	ssc4onnx		Checker with simple ONNX model structure. Simple Structure Checker for ONNX. Analyzes and displays the structure of huge size models that cannot be displayed by Netron.
20	sio4onnx		Simple tool to change the INPUT and OUTPUT shape of ONNX.
21	svs4onnx		A very simple tool to swap connections between output and input variables in an ONNX graph. Simple Variable Switch for ONNX.
22	onnx2tf		Self-Created Tools to convert ONNX files (NCHW) to TensorFlow format (NHWC). The purpose of this tool is to solve the massive Transpose extrapolation problem in onnx-tensorflow (onnx-tf).
23	sng4onnx		A simple tool that automatically generates and assigns an OP name to each OP in an old format ONNX file.
24	sde4onnx		Simple doc_string eraser for ONNX.
25	spo4onnx		Simple tool for partial optimization of ONNX. Further optimize some models that cannot be optimized with onnx-optimizer and onnxsim by several tens of percent. In particular, models containing Einsum and OneHot.
26	components_of_onnx	[WIP]	ONNX parts yard. The various operations described in Operator Schemas are converted in advance into OP stand-alone ONNX files.

2. Very useful tools

No.	Tool Name	Author	Tags	Summary
1	OnnxGraphQt	fateshelled		ONNX model visualizer. Model structure can be edited on the visualization tool.
2	onnx-modifier	ZhangGe6		To edit an ONNX model, One common way is to visualize the model graph, and edit it using ONNX Python API.
3	onnx-simplifier	daquexian		ONNX Simplifier is presented to simplify the ONNX model. It infers the whole computation graph and then replaces the redundant operators with their constant outputs.
4	Sparsify	neuralmagic		Easy-to-use UI for automatically sparsifying neural networks and creating sparsification recipes for better inference performance and a smaller footprint.
5	DeepSparse Engine	neuralmagic		Sparsity-aware neural network inference engine for GPU-class performance on CPUs.
6	Sparsebit	megvii-research		Sparsebit is a toolkit with pruning and quantization capabilities. It is designed to help researchers compress and accelerate neural network models by modifying only a few codes in existing pytorch project.
7	onnion	Idein		onnion project. compile onnx to python. runtime depends only numpy.

2-1. OnnxGraphQt - [WIP] Startup Method Sample

git clone https://github.com/fateshelled/OnnxGraphQt
cd OnnxGraphQt
# build docker image
./docker/build.bash
# run
./docker/run.bash

3. Acknowledgments

1. https://github.com/onnx/onnx/blob/main/docs/PythonAPIOverview.md
2. https://docs.nvidia.com/deeplearning/tensorrt/onnx-graphsurgeon/docs/index.html
3. https://github.com/NVIDIA/TensorRT/tree/main/tools/onnx-graphsurgeon
4. https://github.com/onnx/onnx/blob/main/docs/Operators.md

4. References

1. https://github.com/PINTO0309/PINTO_model_zoo

2. https://github.com/PINTO0309/PINTO_model_zoo/tree/main/115_MoveNet/PINTO_Special/barracuda_gathernd_split

   Kazam_screencast_00062_.mp4

simple-onnx-processing-tools's Issues

sna4onnx: about the param --connection_dest_op_input_names

--connection_dest_op_input_names ADDOP_NAME ADDOP_OUTPUT_NAME DESTOP_NAME DESTOP_INPUT_NAME
Specify the name of the input name from which to connect.
e.g.
-Before-
[OpA]oname1 - iname1[OpB]oname1
[OpC]oname1

  -After-
    [OpA]oname1 - iname1[AddOP1]oname1 - iname1[OpB]oname1
    [OpC]oname1 - iname2[AddOP1]

  When extrapolating a new OP between OpA and OpB.
  --connection_dest_op_input_names AddOP1 oname1 OpB iname1

When I want to add a new OP after [Opc]onname1，how should this parameter be set?
That is to say, if I want to add a new node after the "output" node, how should this parameter be set?

[Query][sbi4onnx][Batch size]modification INPUT batch size will change OUTPUT batch size

Issue Type

Feature Request

Model Name

Fast-BEV onnx model :https://github.com/Sense-GVT/Fast-BEV

ONNX URL

https://github.com/Sense-GVT/Fast-BEV

Description

Hi PINTO0309,
Thanks for your work, first.
I have two question to query
1.I found if we change the INPUT BATCH size as 8, the OUTPUT BATCH size also will change to 8
the command as below
(base) joyoki@joyoki:~/fast-bev$ sbi4onnx -if fast-bev_orig.onnx -of fast-bev_orig_bat ch8.onnx -ics 8
i think this is not right, could you help double check, if we could change INPUT batch size and OUTPUT batch size Separated by need, such as
sbi4onnx -if fast-bev_simply.onnx -ics 8 -of fast-bev_simply_batch6.onnx -ics 1

2.I had checked some document, they both said:Modifying the batch size does not affect accuracy, only affects training speed and efficiency, is right?
orig model:https://drive.google.com/file/d/1gteHTEEJTwwMVYgJxO7zYrnHClL7lkTG/view?usp=share_link

Modified Model:https://drive.google.com/file/d/1x2Z_2psE-oQiqGuFK3OzXwrb0sitMMBE/view?usp=share_link

Thanks

Relevant Log Output

INFO: Finish!,
no error

URL or source code for simple inference testing code

Model link:https://drive.google.com/drive/folders/1NH0BFnBKD7xo6tbWHewSz8frZnxS_7cc?usp=sharing

Convert input and output to NCHW format

Hi @PINTO0309,

Thanks for the great tools.

I tried to use scc4onnx to change the input and output shape from NHWC to NCHW of this ONNX file.

However, I was only able to change the input shape and not the output shape. I was wondering if you can help me change both input and output nodes shape to be in NCHW format.

Thank you!

yolact_edge_onnx_tensorrt_myriad--ONNX EROOR

Hello, thank you for your nice work, I tried your project yolact_edge_onnx_tensorrt_myriad and convert the orignal yolact_edge_54_800000.pth to onnx, when using the onnx for detecting, the onnx out four values, and the model structure is not the same as your example, could you please tell me what is the problem with my operation, thank you very much.
Look forworing your reply

[sne4onnx] Performance Improvement

The following APIs are inefficient and take a very long time to process, so they should be discontinued.

• onnx.utils.Extractor(graph)
  • extractor.extract_model()