InfiniTensor is a high-performance inference engine tailored for GPUs and AI accelerators. Its design focuses on effective deployment and swift academic validation.
make/make build: Builds the project;make install-python: Builds the project then install the python frontend;make test-cpp: Builds the project then run cpp unit tests;make test-onnx: Run python unit tests;
- Sets env:
TEST=OFFto accelerate compiling.- Sets env:
CUDA=ONto enable cuda.- Sets env:
BANG=ONto enable bang.
There are several configurable CMake options, see the CMakeLists.txt file.
- If
USE_BACKTRACEisON,libdw-devhave to be installed. See the README of backward-cpp for details. - If
USE_PROTOBUFisON,protobufhave to be installed. See the README of protobuf for details. - If
USE_CUDAisON,cudahave to be installed.
- RefactorGraph is a newly designed AI framework that is set to replace the current main branch.
- EinNet is going to be merged into the main branch.
- Integration of PET, a tensor program optimizer supporting partially equivalent transformations.
- Supported hardware
- ✔ NVIDIA GPU
- ✔ Cambricon MLU
- ✔ Kunlunxin XPU
- ⬜ Ascend NPU
InfiniTensor development is based on the pull request on Github. Before requesting for merging, a PR should satisfy the following requirements
- Pass all tests.
- Now CI on Github will test everything that can be tested in the ci environment, including code format. So, script
test/script/clang_format_inplace.shis for formatting all code. - Contributors should run
ctestmanually and copy its output to the PR. Use fenced code blocks (triple backquotes, i.e.,```) to avoid referencing in Github. Otherwise,#in the output is interpreted as a Github reference. Do not directly paste the ctest output in commit messages either for the same reason.
- Now CI on Github will test everything that can be tested in the ci environment, including code format. So, script
- Receive at least one approval from reviewers.
- PR title should be concise since it is going to be the commit message in the main branch after merging and squashing.
Please cite EinNet or PET in your publications if it helps your research:
@article{zheng2023einnet,
title={EINNET: Optimizing Tensor Programs with Derivation-Based Transformations},
author={Zheng, Liyan and Wang, Haojie and Zhai, Jidong and Hu, Muyan and Ma, Zixuan and Wang, Tuowei and Huang, Shuhong and Miao, Xupeng and Tang, Shizhi and Huang, Kezhao and Jia, Zhihao},
booktitle={17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23)},
pages={739--755},
year={2023}
}
@inproceedings{wang2021pet,
title={PET: Optimizing tensor programs with partially equivalent transformations and automated corrections},
author={Wang, Haojie and Zhai, Jidong and Gao, Mingyu and Ma, Zixuan and Tang, Shizhi and Zheng, Liyan and Li, Yuanzhi and Rong, Kaiyuan and Chen, Yuanyong and Jia, Zhihao},
booktitle={15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21)},
pages={37--54},
year={2021}
}
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent