haobosang / tinytensor

TinyTensor is a tool for running already trained NN (Neural Network) models to be able to use them for inference of various tasks such as image classification, semantic segmentation, etc.

TinyTensor is an efficient lightweight deep learning inference framework.

About

TinyTensor supports a variety of popular neural network architectures such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), and fully connected neural networks, and it can be used for tasks such as image classification, object detection, speech recognition, and natural language processing.

Development Environment

• Development language: C++ 20
• Math Library: Armadillo
• Logging framework：Google glog
• Unit test: Google Test
• Code style: Clang format
• Performance testing： Benckmark

How to build on Linux

Ubuntu 18 (Debian 10)

apt update
apt install cmake libopenblas-dev liblapack-dev \
libarpack2-dev libsuperlu-dev libomp-dev libopencv-dev

Install Armadillo

wget https://sourceforge.net/projects/arma/files/armadillo-12.2.0.tar.xz
mkdir build && cd build
cmake ..
make -j8
make install

Install Benchmark

cd third_party
git submodule update --init
mv googletest benchmark
mkdir build && cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE ../benchmark
make -j8
# 如果想全局安装就接着运行下面的命令
sudo make install

Operators Currently Implemented

• ReLU
• Sigmoid
• Conv
• MaxPooling

Performance Testing

Test Equipment

Intel(R) Xeon(R) W-2223 CPU @ 3.60GHz

Compilation Environment

gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0

Performance Results

Input size	Model	Computing Device	Time
224×224 batch = 8	ResNet18	CPU(armadillo)	55ms / image
224×224 batch =16	ResNet18	CPU(armadillo)	28.5ms / image

Acknowledgement

caffe

tinytensor's Issues

用Eigen重新实现Tensor

Sigmoid operator implementation

链接openblas或intel-mkl提高模型推理速度

Linear operator implementation

adaptive average pooling operator implementation

sigmoid代码问题

Flatten operator implementation

Maxpooling operator implementation