English | 简体中文
⚡️FastDeploy is an accessible and efficient deployment Development Toolkit. It covers 🔥critical AI models in the industry and provides 📦out-of-the-box deployment experience. It covers image classification, object detection, image segmentation, face detection, face recognition, human keypoint detection, OCR, semantic understanding and other tasks to meet developers' industrial deployment needs for multi-scenario, multi-hardware and multi-platform .
Potrait Segmentation | Image Matting | Semantic Segmentation | Real-Time Matting |
---|---|---|---|
![]() |
![]() |
![]() |
![]() |
OCR | Behavior Recognition | Object Detection | Pose Estimation |
![]() |
![]() |
![]() |
![]() |
Face Alignment | 3D Object Detection | Face Editing | Image Animation |
![]() |
![]() |
![]() |
![]() |
- 🔥 2022.8.18:Release FastDeploy release/v0.2.0
- New server-side deployment upgrade: faster inference performance, support more vision model
- Release high-performance inference engine SDK based on x86 CPUs and NVIDIA GPUs, with significant increase in inference speed
- Integrate Paddle Inference, ONNXRuntime, TensorRT and other inference engines and provide a seamless deployment experience
- Supports full range of object detection models such as YOLOv7, YOLOv6, YOLOv5, PP-YOLOE and provides End-To-End Deployment Demos
- Support over 40 key models and Demo Examples including face detection, face recognition, real-time portrait matting, image segmentation.
- Support deployment in both Python and C++
- Supports Rockchip, Amlogic, NXP and other NPU chip deployment capabilities on edge device deployment
- Release Lightweight Object Detection Picodet-NPU Deployment Demo, providing the full quantized inference capability for INT8.
- New server-side deployment upgrade: faster inference performance, support more vision model
- Data Center and Cloud Deployment
- Mobile and Edge Device Deployment
- Community
- Acknowledge
- License
- CUDA >= 11.2
- cuDNN >= 8.0
- python >= 3.6
- OS: Linux x86_64/macOS/Windows 10
pip install fastdeploy-gpu-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
conda config --add channels conda-forge && conda install cudatoolkit=11.2 cudnn=8.2
pip install fastdeploy-python -f https://www.paddlepaddle.org.cn/whl/fastdeploy.html
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
# For deployment of GPU/TensorRT, please refer to examples/vision/detection/paddledetection/python
import cv2
import fastdeploy.vision as vision
model = vision.detection.PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml")
im = cv2.imread("000000014439.jpg")
result = model.predict(im.copy())
print(result)
vis_im = vision.vis_detection(im, result, score_threshold=0.5)
cv2.imwrite("vis_image.jpg", vis_im)
- Please refer to C++ Prebuilt Libraries Download
- Prepare models and pictures
wget https://bj.bcebos.com/paddlehub/fastdeploy/ppyoloe_crn_l_300e_coco.tgz
tar xvf ppyoloe_crn_l_300e_coco.tgz
wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000014439.jpg
- Test inference results
// For GPU/TensorRT deployment, please refer to examples/vision/detection/paddledetection/cpp
#include "fastdeploy/vision.h"
int main(int argc, char* argv[]) {
namespace vision = fastdeploy::vision;
auto model = vision::detection::PPYOLOE("ppyoloe_crn_l_300e_coco/model.pdmodel",
"ppyoloe_crn_l_300e_coco/model.pdiparams",
"ppyoloe_crn_l_300e_coco/infer_cfg.yml");
auto im = cv::imread("000000014439.jpg");
vision::DetectionResult res;
model.Predict(&im, &res);
auto vis_im = vision::Visualize::VisDetection(im, res, 0.5);
cv::imwrite("vis_image.jpg", vis_im);
return 0;
}
For more deployment models, please refer to Vision Model Deployment Examples .
Notes:
✅: already supported; ❔: to be supported in the future; ❌: not supported now;
Task | Model | API | Linux | Linux | Win | Win | Mac | Mac | Linux | Linux |
---|---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | X86 CPU | NVIDIA GPU | Intel CPU | NVIDIA GPU | Intel CPU | Arm CPU | AArch64 CPU | NVIDIA Jetson |
Classification | PaddleClas/ResNet50 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-LCNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-LCNetv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/EfficientNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/GhostNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/MobileNetV3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/ShuffleNetV2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SqueeezeNetV1.1 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/Inceptionv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/PP-HGNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Classification | PaddleClas/SwinTransformer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PP-YOLOE | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PicoDet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/YOLOv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | PaddleDetection/PP-YOLO | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
Detection | PaddleDetection/PP-YOLOv2 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
Detection | PaddleDetection/FasterRCNN | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ |
Detection | Megvii-BaseDetection/YOLOX | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | WongKinYiu/YOLOv7 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | meituan/YOLOv6 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | ultralytics/YOLOv5 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | WongKinYiu/YOLOR | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | WongKinYiu/ScaledYOLOv4 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | ppogg/YOLOv5Lite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Detection | RangiLyu/NanoDetPlus | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/PP-LiteSeg | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/PP-HumanSegLite | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/HRNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/PP-HumanSegServer | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/Unet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Segmentation | PaddleSeg/Deeplabv3 | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceDetection | biubug6/RetinaFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceDetection | Linzaer/UltraFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceDetection | deepcam-cn/YOLOv5Face | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceDetection | deepinsight/SCRFD | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceRecognition | deepinsight/ArcFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceRecognition | deepinsight/CosFace | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceRecognition | deepinsight/PartialFC | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
FaceRecognition | deepinsight/VPL | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
Matting | ZHKKKe/MODNet | Python/C++ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
- ARM Linux System
Model | Size (MB) | Linux | Android | iOS | Linux | Linux | Linux | TBD... | |
---|---|---|---|---|---|---|---|---|---|
--- | --- | --- | ARM CPU | ARM CPU | ARM CPU | Rockchip-NPU RV1109 RV1126 RK1808 |
Amlogic-NPU A311D S905D C308X |
NXPNPU i.MX 8M Plus |
TBD...| |
Classification | PP-LCNet | 11.9 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | PP-LCNetv2 | 26.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | EfficientNet | 31.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | GhostNet | 20.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV1 | 17 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV2 | 14.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | MobileNetV3 | 22 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Classification | ShuffleNetV2 | 9.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | SqueezeNetV1.1 | 5 | ✅ | ✅ | ✅ | ||||
Classification | Inceptionv3 | 95.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | PP-HGNet | 59 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Classification | SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ❔ |
Detection | CenterNet | 4.8 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | YOLOv3_MobileNetV3 | 94.6 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_ResNet50vd | 188.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Detection | YOLOv5s | 29.3 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Face Detection | BlazeFace | 1.5 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Face Detection | RetinaFace | 1.7 | ✅ | ❌ | ❌ | -- | -- | -- | -- |
Keypoint Detection | PP-TinyPose | 5.5 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
Segmentation | PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | PP-HumanSeg-Server | 107.2 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
Segmentation | Unet | 53.7 | ❌ | ✅ | ❌ | -- | -- | -- | -- |
OCR | PP-OCRv1 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
OCR | PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
OCR | PP-OCRv3 | 2.4+10.6 | ✅ | ✅ | ✅ | ❔ | ❔ | ❔ | ❔ |
OCR | PP-OCRv3-tiny | 2.4+10.7 | ✅ | ✅ | ✅ | -- | -- | -- | -- |
- If you have any question or suggestion, please give us your valuable input via GitHub Issues
- Join Us👬: Scan the QR code via WeChat to join our FastDeploy technology communication group
We sincerely appreciate the open-sourced capabilities in EasyEdge as we adopt it for the SDK generation and download in this project.
FastDeploy is provided under the Apache-2.0.