TensorRT8.Support Yolov5n,s,m,l,x .darknet -> tensorrt. Yolov4 Yolov3 use raw darknet *.weights and *.cfg fils. If the wrapper is useful to you,please Star it.
License: MIT License
需要自己加上些图像吗？
@enazoe 您好,
感谢您的无私贡献.
1.环境
Ubuntu 18.04,
1060TI,
Tensorrt 7,
CUDA 10.0,
Opencv 3.3.1
2.问题:
我在本地pc测试sample_detector, 用的默认的FP32, yolov4:
sample_detector的inference在16fps.
darknet yolov4的inference也在15fps
感觉相差不大.请问是不是哪里不正确?感谢
$ ./yolo-trt
WARNING: couldn't find: data/test/07/0.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/1.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/2.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/3.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/4.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/5.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/6.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/7.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/8.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/9.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/10.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/11.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/12.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/13.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/14.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/15.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/16.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/17.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/18.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/19.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/20.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/21.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/22.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/23.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/24.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/25.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/26.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/27.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/28.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/29.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/07/30.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/0.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/1.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/2.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/3.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/4.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/5.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/6.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/7.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/8.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/9.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/10.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/11.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/12.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/13.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/14.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/15.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/16.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/17.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/18.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/19.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/20.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/21.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/22.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/23.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/24.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/25.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/26.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/27.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/28.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/29.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/08/30.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/0.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/1.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/2.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/3.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/4.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/5.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/6.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/7.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/8.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/9.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/10.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/11.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/12.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/13.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/14.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/11/15.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/0.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/1.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/2.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/3.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/4.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/5.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/6.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/7.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/8.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/9.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/10.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/11.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/12.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/13.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/14.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/12/15.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/0.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/1.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/2.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/3.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/4.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/5.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/6.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/7.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/8.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/9.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/10.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/11.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/12.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/13.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/14.jpg while loading: ../configs/calibration_images.txt
WARNING: couldn't find: data/test/13/15.jpg while loading: ../configs/calibration_images.txt
Loading pre-trained weights...
Loading complete!
layer inp_size out_size weightPtr
(1) conv-bn-leaky 3 x 416 x 416 16 x 416 x 416 496
(2) maxpool 16 x 416 x 416 16 x 208 x 208 496
(3) conv-bn-leaky 16 x 208 x 208 32 x 208 x 208 5232
(4) maxpool 32 x 208 x 208 32 x 104 x 104 5232
(5) conv-bn-leaky 32 x 104 x 104 64 x 104 x 104 23920
(6) maxpool 64 x 104 x 104 64 x 52 x 52 23920
(7) conv-bn-leaky 64 x 52 x 52 128 x 52 x 52 98160
(8) maxpool 128 x 52 x 52 128 x 26 x 26 98160
(9) conv-bn-leaky 128 x 26 x 26 256 x 26 x 26 394096
(10) maxpool 256 x 26 x 26 256 x 13 x 13 394096
(11) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 1575792
(12) maxpool 512 x 13 x 13 512 x 13 x 13 1575792
(13) conv-bn-leaky 512 x 13 x 13 1024 x 13 x 13 6298480
(14) conv-bn-leaky 1024 x 13 x 13 256 x 13 x 13 6561648
(15) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 7743344
(16) conv-linear 512 x 13 x 13 255 x 13 x 13 7874159
(17) yolo 255 x 13 x 13 255 x 13 x 13 7874159
(18) route - 256 x 13 x 13 7874159
(19) conv-bn-leaky 256 x 13 x 13 128 x 13 x 13 7907439
(20) upsample 128 x 13 x 13 128 x 26 x 26 -
(21) route - 384 x 26 x 26 7907439
(22) conv-bn-leaky 384 x 26 x 26 256 x 26 x 26 8793199
(23) conv-linear 256 x 26 x 26 255 x 26 x 26 8858734
(24) yolo 255 x 26 x 26 255 x 26 x 26 8858734
File does not exist : ../configs/yolov3-tiny-kINT8.engine
Building the TensorRT Engine...
New calibration table will be created to build the engine
WARNING: Unable to open image : data/test/12/14.jpg
yolo-trt: /home/jiale/yolo-tensorrt/modules/ds_image.cpp:106: DsImage::DsImage(const string&, const int&, const int&): Assertion `0' failed.
Aborted (core dumped)
Where could I get data/test/...
just like the title.
Hi,
After executing the compiled binaries on Ubuntu, I hit this error :
Cuda failure: no kernel image is available for execution on the device in file /home/path/to//TensorRT/yolo-tensorrt/modules/plugin_factory.cpp at line 155 Aborted (core dumped)
Any idea on why does it occur ?
Ubuntu 18.04
Cuda-10.2
TRT 7.5
Hello, I got this error in compiling :
../../src_file/libdetector.so: undefined reference to `cv::dnn::dnn4_v20190621::blobFromImages(cv::InputArray const&, double, cv::Size, cv::Scalar_ const&, bool, bool, int)'
I think that I am linking wrong the library in my own project, how can I link correctly?
I am using a MakeFile not cmake...Can you give me an example of how to import and use the library and use it in your project?
Some questions.
In yolo.cpp you set condition: assert(m_InputW == m_InputH);
Why? AlexeyAB write: set network size width=416 height=416 or any value multiple of 32
And darknet trains and works with input size 960x768
And second question about this assert:
if (weights.size() != weightPtr)
{
std::cout << "Number of unused weights left : " << weights.size() - weightPtr << std::endl;
assert(0);
}
In my yolov4-tiny with size 608x608 this condition goes to assert but in release it works. And in my case weights.size() < weightPtr
抱歉还要打扰一下，我还有个小白问题，之前没做过tensorRT。我看到如果没有engine文件，程序就会根据cfg和weights来默认生成一个engine文件，下次启动时如果已有engine文件就可以直接使用，我看到engine文件也足够的大，超过了weights的大小，我的问题是：
engine文件是否已经包含了cfg和weights的所有信息，以便一旦生成engine文件后就不再需要cfg和weights文件了，这样的话模型的整体占用空间就会少一半，因为毕竟yolov3/4/5原始的模型还是很大的；还是说每次启动时cfg和weights是必须的规避不了？我现在看到的现象是如果没有找到cfg和weights，程序就跑飞了
Hello I got this error while making the code.
/usr/bin/ld: cannot find -lgflags
collect2: error: ld returned 1 exit status
CMakeFiles/detector.dir/build.make:1039: recipe for target 'libdetector.so' failed
make[2]: *** [libdetector.so] Error 1
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/detector.dir/all' failed
make[1]: *** [CMakeFiles/detector.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
I read that " The project generate the libdetector.so lib, and the sample code. If you want to use the generated libdetector.so lib in your own project,the cmake file perhaps could help you in scripts dir."
I don't know if it is related to my problem... I am a beginner of jetson and deepstream platforms.
Thanks for your help
一直有这个错误，求问怎么解决呢？
严重性 代码 说明 项目 文件 行 禁止显示状态
错误 LNK2001 无法解析的外部符号 "enum cudaError __cdecl cudaYoloLayerV3(void const *,void *,unsigned int const &,unsigned int const &,unsigned int const &,unsigned int const &,unsigned __int64,struct CUstream_st *)" (?cudaYoloLayerV3@@ya?AW4cudaError@@PEBXPEAXAEBI222_KPEAUCUstream_st@@@z) tensorRT_test C:\Users\NYBC-LAB\Desktop\tensorRT_test\tensorRT_test\plugin_factory.obj 1
I saw YoloV4 was published a few days ago. Could you implement YoloV4 to use it with TensorRT?
Thank you for your time.
Config config_v3;
config_v3.net_type = YOLOV3_TINY;
config_v3.file_model_cfg = "../configs/yolov3-tiny.cfg";
config_v3.file_model_weights = "../configs/yolov3-tiny.weights";
config_v3.calibration_image_list_file_txt = "../configs/calibration_images.txt";
config_v3.inference_precison = INT8;
yolov3, yolov4 and yolov4-tiny run successfully.
but yolov3-tiny failed.
Loading pre-trained weights...
Loading complete!
layer inp_size out_size weightPtr
(1) conv-bn-leaky 3 x 416 x 416 16 x 416 x 416 496
(2) maxpool 16 x 416 x 416 16 x 208 x 208 496
(3) conv-bn-leaky 16 x 208 x 208 32 x 208 x 208 5232
(4) maxpool 32 x 208 x 208 32 x 104 x 104 5232
(5) conv-bn-leaky 32 x 104 x 104 64 x 104 x 104 23920
(6) maxpool 64 x 104 x 104 64 x 52 x 52 23920
(7) conv-bn-leaky 64 x 52 x 52 128 x 52 x 52 98160
(8) maxpool 128 x 52 x 52 128 x 26 x 26 98160
(9) conv-bn-leaky 128 x 26 x 26 256 x 26 x 26 394096
(10) maxpool 256 x 26 x 26 256 x 13 x 13 394096
(11) conv-bn-leaky 256 x 13 x 13 512 x 13 x 13 1575792
(12) maxpool 512 x 13 x 13 512 x 12 x 12 1575792
(13) conv-bn-leaky 512 x 12 x 12 1024 x 12 x 12 6298480
(14) conv-bn-leaky 1024 x 12 x 12 256 x 12 x 12 6561648
(15) conv-bn-leaky 256 x 12 x 12 512 x 12 x 12 7743344
(16) conv-linear 512 x 12 x 12 255 x 12 x 12 7874159
(17) yolo 255 x 12 x 12 255 x 12 x 12 7874159
(18) route - 256 x 12 x 12 7874159
(19) conv-bn-leaky 256 x 12 x 12 128 x 12 x 12 7907439
(20) upsample 128 x 12 x 12 128 x 24 x 24 -
ERROR: Assertion failed: d.nbDims >= 1, file f:\image-process-lib\code\tensorrt-7.0.0.11\samples\yolo-tensorrt\modules\trt_utils.cpp, line 409
Hi,
I was trying to build and run the sample on Ubuntu 18.04 and encountered the following error. Was wondering if anyone tried something similar?
[build] ./yolov3-trt master
Loading pre-trained weights...
Loading complete!
Using previously generated plan file located at ../configs/yolov3-kFLOAT.engine
Loading TRT Engine...
Loading Complete!
Cuda failure: no kernel image is available for execution on the device in file /work/code/yolo-tensorrt/modules/plugin_factory.cpp at line 155
[1] 19770 abort (core dumped) ./yolov3-trt
I have followed the prerequisites listed in the main read me.
求助作者大大和各位大佬~
xavier上编译项目成功，但运行demo的时候报错：
modules/yolo.cpp:522: void Yolo::createYOLOEngine(nvinfer1::DataType, Int8EntropyCalibrator*): Assertion `m_Engine != nullptr' failed.
系统环境：
JetPack 4.3
Ubuntu 18.04.3 LTS
CUDA 10.0.326
cudnn 7.6.3
Tensorrt 6.0.1
OpenCV 4.1.1
单线程调用正常，多线程调用时跑一会就会报类似Cuda Error in execute: 77 (an illegal memory access was encountered)的错误，错误处大约在yolo.cpp 中的Yolo::doInference
NV_CUDA_CHECK(cudaMemcpyAsync(m_DeviceBuffers.at(m_InputBindingIndex), input, batchSize * m_InputSize * sizeof(float), cudaMemcpyHostToDevice, m_CudaStream));
有遇到相关的问题吗？
Hello, I have a question for "libnvinfer.so:对'[email protected]'未定义的引用", the version is ubuntu=1804, cuda=10.0, tensorrt=5.1.5, cudnn=7.5.0, opencv=3.4, can you help me for this question? It is a question for version or coding ?
undefined symbol: _ZN8Detector6detectERKN2cv3MatERSt6vectorI6ResultSaIS5_EE
undefined symbol: _ZN8Detector4initERK6Config
I could run your sample yesterday.But today when I ran this sample,the errors above happened.I try to solve it,but it always show these errors.I know that the undefined symbol happens in "detector_->init(config)" and "detector_->detect()".My device is Nvidia Xavier NX,and the os is Ubuntu 18.04 customed by Nvidia.
您好，我想修改测试的batchsize，只是修改_p_net = std::unique_ptr{ new YoloV3(1, _yolo_info, _infer_param) };这里好像不行，请问还有别的地方需要修改吗 @enazoe
Hi @enazoe ,
What is the usage of yoloplugin_lib.h and yoloplugin_lib.cpp?
It seems for batch inference?
What's the preprocess code for yolov4 ?
Thanks!
博主，你好。目前我已经在windows上将yolov4编译成功了。
环境：win10, tensorrt6.0.1.5, cuda10.0, cudnn7.6.5, 1080Ti。
目前我针对自己训练的模型进行了测速。
配置文件中图片大小为：800x800x3，tensorrt精度为FP16，batchsize为1。
enquequ()+cudaMemcpyAsync()的时间为1ms，但是cudaStreamSynchronize()操作花费了29ms，请问这个地方有没有能够改善了方法，非常感谢。
博主，你好。请问该项目支持在tensorrt6.0.1.5下编译吗？win10+tensorrt6.0.1.5+cuda10.0+cudnn7.6.5编译出错了。
Hello,
I am trying to run the TensorRT executable "yolo-trt.exe" on Windows 10 with
CUDA 11.0.3, TensorRT 7.1.3.4 and cudnn 8.0.3 and I get the error below.
Do you have any idea what could be causing the error? I tried both Yolo V3 and V4 configs
and the error is the same.
Any help is appreciated.
Thanks!
(157) conv-bn-leaky 1024 x 13 x 13 512 x 13 x 13 54196318
(158) conv-bn-leaky 512 x 13 x 13 1024 x 13 x 13 58919006
(159) conv-bn-leaky 1024 x 13 x 13 512 x 13 x 13 59445342
(160) conv-bn-leaky 512 x 13 x 13 1024 x 13 x 13 64168030
(161) conv-linear 1024 x 13 x 13 255 x 13 x 13 64429405
(162) yolo 255 x 13 x 13 255 x 13 x 13 64429405
File does not exist : ../configs/yolov4-kFLOAT-batch1.engine
Building the TensorRT Engine...
Building complete!
Serializing the TensorRT Engine...
Serialized plan file cached at location : ../configs/yolov4-kFLOAT-batch1.engine
Loading TRT Engine...
Loading Complete!
WARNING: Current optimization profile is: 0. Please ensure there are no enqueued operations pending in this context prior to switching profiles
Cuda failure: invalid argument in file C:\git\yolo-tensorrt\modules\yolo.cpp at line 925
(pytorch_opencv) PS C:\git\yolo-tensorrt\Release>
@enazoe Did you evaluate the detection precision? Too much accuracy reduction compared to no acceleration.
Hi!
After latest update yolov4-tiny works fine but yolov4 failed:
Loading pre-trained weights...
Loading complete!
layer inp_size out_size weightPtr
(1) conv-bn-mish 3 x 608 x 608 32 x 608 x 608 992
(2) conv-bn-mish 32 x 608 x 608 64 x 304 x 304 19680
(3) conv-bn-mish 64 x 304 x 304 64 x 304 x 304 24032
(4) route - 64 x 304 x 304 24032
(5) conv-bn-mish 64 x 304 x 304 64 x 304 x 304 28384
(6) conv-bn-mish 64 x 304 x 304 32 x 304 x 304 30560
(7) conv-bn-mish 32 x 304 x 304 64 x 304 x 304 49248
(8) skip 64 x 304 x 304 64 x 304 x 304 -
(9) conv-bn-mish 64 x 304 x 304 64 x 304 x 304 53600
(10) route - 128 x 304 x 304 53600
(11) conv-bn-mish 128 x 304 x 304 64 x 304 x 304 62048
(12) conv-bn-mish 64 x 304 x 304 128 x 152 x 152 136288
(13) conv-bn-mish 128 x 152 x 152 64 x 152 x 152 144736
(14) route - 128 x 152 x 152 144736
(15) conv-bn-mish 128 x 152 x 152 64 x 152 x 152 153184
(16) conv-bn-mish 64 x 152 x 152 64 x 152 x 152 157536
(17) conv-bn-mish 64 x 152 x 152 64 x 152 x 152 194656
(18) skip 64 x 152 x 152 64 x 152 x 152 -
(19) conv-bn-mish 64 x 152 x 152 64 x 152 x 152 199008
(20) conv-bn-mish 64 x 152 x 152 64 x 152 x 152 236128
(21) skip 64 x 152 x 152 64 x 152 x 152 -
(22) conv-bn-mish 64 x 152 x 152 64 x 152 x 152 240480
(23) route - 128 x 152 x 152 240480
(24) conv-bn-mish 128 x 152 x 152 128 x 152 x 152 257376
(25) conv-bn-mish 128 x 152 x 152 256 x 76 x 76 553312
(26) conv-bn-mish 256 x 76 x 76 128 x 76 x 76 586592
(27) route - 256 x 76 x 76 586592
(28) conv-bn-mish 256 x 76 x 76 128 x 76 x 76 619872
(29) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 636768
(30) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 784736
(31) skip 128 x 76 x 76 128 x 76 x 76 -
(32) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 801632
(33) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 949600
(34) skip 128 x 76 x 76 128 x 76 x 76 -
(35) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 966496
(36) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1114464
(37) skip 128 x 76 x 76 128 x 76 x 76 -
(38) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1131360
(39) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1279328
(40) skip 128 x 76 x 76 128 x 76 x 76 -
(41) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1296224
(42) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1444192
(43) skip 128 x 76 x 76 128 x 76 x 76 -
(44) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1461088
(45) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1609056
(46) skip 128 x 76 x 76 128 x 76 x 76 -
(47) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1625952
(48) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1773920
(49) skip 128 x 76 x 76 128 x 76 x 76 -
(50) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1790816
(51) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1938784
(52) skip 128 x 76 x 76 128 x 76 x 76 -
(53) conv-bn-mish 128 x 76 x 76 128 x 76 x 76 1955680
(54) route - 256 x 76 x 76 1955680
(55) conv-bn-mish 256 x 76 x 76 256 x 76 x 76 2022240
(56) conv-bn-mish 256 x 76 x 76 512 x 38 x 38 3203936
(57) conv-bn-mish 512 x 38 x 38 256 x 38 x 38 3336032
(58) route - 512 x 38 x 38 3336032
(59) conv-bn-mish 512 x 38 x 38 256 x 38 x 38 3468128
(60) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 3534688
(61) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 4125536
(62) skip 256 x 38 x 38 256 x 38 x 38 -
(63) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 4192096
(64) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 4782944
(65) skip 256 x 38 x 38 256 x 38 x 38 -
(66) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 4849504
(67) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 5440352
(68) skip 256 x 38 x 38 256 x 38 x 38 -
(69) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 5506912
(70) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 6097760
(71) skip 256 x 38 x 38 256 x 38 x 38 -
(72) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 6164320
(73) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 6755168
(74) skip 256 x 38 x 38 256 x 38 x 38 -
(75) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 6821728
(76) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 7412576
(77) skip 256 x 38 x 38 256 x 38 x 38 -
(78) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 7479136
(79) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 8069984
(80) skip 256 x 38 x 38 256 x 38 x 38 -
(81) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 8136544
(82) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 8727392
(83) skip 256 x 38 x 38 256 x 38 x 38 -
(84) conv-bn-mish 256 x 38 x 38 256 x 38 x 38 8793952
(85) route - 512 x 38 x 38 8793952
(86) conv-bn-mish 512 x 38 x 38 512 x 38 x 38 9058144
(87) conv-bn-mish 512 x 38 x 38 1024 x 19 x 19 13780832
(88) conv-bn-mish 1024 x 19 x 19 512 x 19 x 19 14307168
(89) route - 1024 x 19 x 19 14307168
(90) conv-bn-mish 1024 x 19 x 19 512 x 19 x 19 14833504
(91) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 15097696
(92) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 17459040
(93) skip 512 x 19 x 19 512 x 19 x 19 -
(94) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 17723232
(95) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 20084576
(96) skip 512 x 19 x 19 512 x 19 x 19 -
(97) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 20348768
(98) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 22710112
(99) skip 512 x 19 x 19 512 x 19 x 19 -
(100) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 22974304
(101) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 25335648
(102) skip 512 x 19 x 19 512 x 19 x 19 -
(103) conv-bn-mish 512 x 19 x 19 512 x 19 x 19 25599840
(104) route - 1024 x 19 x 19 25599840
(105) conv-bn-mish 1024 x 19 x 19 1024 x 19 x 19 26652512
(106) conv-bn-leaky 1024 x 19 x 19 512 x 19 x 19 27178848
(107) conv-bn-leaky 512 x 19 x 19 1024 x 19 x 19 31901536
(108) conv-bn-leaky 1024 x 19 x 19 512 x 19 x 19 32427872
(109) maxpool 512 x 19 x 19 512 x 15 x 15 32427872
(110) route - 512 x 19 x 19 32427872
(111) maxpool 512 x 19 x 19 512 x 11 x 11 32427872
(112) route - 512 x 19 x 19 32427872
(113) maxpool 512 x 19 x 19 512 x 7 x 7 32427872
ERROR: ERROR: (114) route - 32567 32427872
ERROR: ERROR: ERROR: (115) conv-bn-leaky 32567 32567 33478496
ERROR: ERROR: ERROR: ERROR: (116) conv-bn-leaky 32567 32567 33482592
ERROR: ERROR: ERROR: ERROR: (117) conv-bn-leaky 32567 32567 33484640
ERROR: ERROR: ERROR: ERROR: (118) conv-bn-leaky 32567 32567 33485664
ERROR: ERROR: ERROR: Segmentation fault (core dumped)
Hello i got this error:
Building the TensorRT Engine...
ERROR: ERROR: yolo-trt: /home/nvidia/Documents/yolo-tensorrt/modules/yolo.cpp:522: void Yolo::createYOLOEngine(nvinfer1::DataType, Int8EntropyCalibrator*): Assertion `m_Engine != nullptr' failed.
Aborted (core dumped)
I am using jetpack 4.3 , CUDA : 10.0.326 , OpencCV : 4.1.2, TensorRT: 6.0.1.10 on nano
Any suggest ??
您好，我在jetson nano平台上测试了yolov3-tiny和yolov5的性能
| model | input size | batchsize | precision | inference time |
|---|---|---|---|---|
| yolov3-tiny | 416x416 | 2 | FP16 | ~60ms |
| yolov5s-3.0 | 320x320 | 2 | FP16 | ~120ms |
| yolov5m-3.0 | 640x320 | 2 | FP16 | ~230ms |
有个疑问，为什么yolov5s和yolov5m模型比yolov3-tiny小，而推理速度确更慢？
之前我也通过python 测试过yolov5s（pytorch->onnx->tensorrt），而推理耗时在100ms左右，比这个C++的版本快些。
您好，您提供的yaml2cfg.py转换后的cfg和weights模型是不是只能用于tensorRT的推理，是不能被AlexeyAB darknet的repo所训练的对吗
作者您好:
关于解析cfg和weights, 看您是自己写的函数,
如果用darknetAB自己的parse_network_cfg(...)和load_weights(...)进行解析可以吗?您之前是如何考虑的呢?
谢谢!
I'm trying to load yolov5s as in the samples, but getting this error. Output is something like
Loading pre-trained weights...
Loading complete!
layer inp_size out_size
(1) Focus 3 x 320 x 640 32 x 160 x 320
(2) Conv 32 x 160 x 320 64 x 80 x 160
(3) BottleneckCSP 64 x 80 x 160 64 x 80 x 160
(4) Conv 64 x 80 x 160 128 x 40 x 80
(5) BottleneckCSP 128 x 40 x 80 128 x 40 x 80
(6) Conv 128 x 40 x 80 256 x 20 x 40
(7) BottleneckCSP 256 x 20 x 40 256 x 20 x 40
(8) Conv 256 x 20 x 40 512 x 10 x 20
(9) SPP 512 x 10 x 20 512 x 10 x 20
terminate called after throwing an instance of 'std::invalid_argument'
terminate called recursively
And gdb states that it crashes in
Yolo::create_engine_yolov5() ---> parse_bottleneck_args() ----> std::__cxx11::stoi()
What might be the issue? Can you help?
Hi!
Do you have plans for YOLO v4 support?
Hi!
I'm build your project with extra compiler flags: -Wall -Wextra -pedantic-errors
And the build was broke. And I create one fix: #43
@enazoe 作者您好,
关于yolov4-tiny部分, 我想把IPluginV2IOExt 换成IPluginV2, 然后支持tensorrt5. 如下:
trt7:
class Chunk : public IPluginV2IOExt{...}trt5:
class Chunk : public IPluginV2{...}请问这样换完之后(版本暂且成为trt5), 是不是需要用trt5生成新的engine之后, 才能做推断.
trt5直接加载 原始的trt7的已经转好的模型 推断会不会有问题呢?
谢谢!
使用yolov3-tiny 进行测试，GPU：1050 ，单张测试图片检测速度 竟然是12221ms， 明显感觉不对，是什么其他地方要注意吗？
Loading pre-trained weights...
Loading complete!
Using previously generated plan file located at ../configs/yolov3-tiny-kFLOAT.engine
Loading TRT Engine...
Loading Complete!
Label:2
className:car
Label:7
className:truck
Label:16
className:dog
detect time = 12221ms
id:2 prob:0.725098 rect:[220 x 89 from (466, 82)]
id:7 prob:0.584577 rect:[99 x 67 from (530, 94)]
id:16 prob:0.831854 rect:[257 x 299 from (124, 218)]
Thank you very much for sharing the project， but I have a error transferring to yolov4. It is show mistake "Internal error: could not find any implementation for node (Unnamed Layer* 271) [Deconvolution], try increasing the workspace size with IBuilder:: setMaxWorkspaceSize() ../builder/tacticOptimizer.cpp (1523) - OutOfMemory Error in computeCosts: 0",
Please help me to answer， thanks very much
hello. I have some problems with sln for windows. Could you help to fix it? I have already changed all custom dependencies but still get some errors and I don't know how to handle with it.
I have also several warnings (30). Most of them about conversion like "size_t" to "unit32_t" or initialization like "double" to "const float"
Could you help me to fix my issue? Thank you!
作者您好,
我看您是把所有层都解析完成之后, 才判断是否有已经产生好的engine. 可不可以把这个判断放在最前面?
yolo-tensorrt/modules/yolo.cpp
Line 475 in 91455c6
我想直接读取保存好的engine 然后进行推断. 也就是直接
detector_->detect(mat_temp, res);
而不进行初始化去创建engine
detector_->init(config_v4_tiny);
但是这样做是有错的
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
请问这里我应该如何实现比较好?
感谢!
Hi I download your code in Windows using
OpenCV 4..3.0, cuda 11.0, tensorrt TensorRT-7.1.3.4
Build is fine, but when I would like to convert yolov3. it casue problems in
nvinfer1::IBuilderConfig* config = m_Builder->createBuilderConfig();
Do you have any suggestions?
I have installed opencv on my jetson nano. But I get this error:
stiv@nano:~/yolo-tensorrt/build$ cmake ..
-- gflags found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda (found version "10.0")
-- CUDA version:$(CUDA_VERSION)
-- CUDA 10 detected
-- Found OpenCV: /usr/local (found version "4.1.2")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/stiv/yolo-tensorrt/build
stiv@nano:~/yolo-tensorrt/build$ make
[ 7%] Building NVCC (Device) object CMakeFiles/detector.dir/modules/detector_generated_kernel.cu.o
Scanning dependencies of target detector
[ 14%] Building CXX object CMakeFiles/detector.dir/modules/calibrator.cpp.o
In file included from /home/stiv/yolo-tensorrt/modules/ds_image.h:28:0,
from /home/stiv/yolo-tensorrt/modules/calibrator.h:29,
from /home/stiv/yolo-tensorrt/modules/calibrator.cpp:26:
/home/stiv/yolo-tensorrt/modules/trt_utils.h:32:10: fatal error: opencv2/opencv.hpp: No such file or directory
#include <opencv2/opencv.hpp>
^~~~~~~~~~~~~~~~~~~~
compilation terminated.
CMakeFiles/detector.dir/build.make:69: recipe for target 'CMakeFiles/detector.dir/modules/calibrator.cpp.o' failed
make[2]: *** [CMakeFiles/detector.dir/modules/calibrator.cpp.o] Error 1
CMakeFiles/Makefile2:104: recipe for target 'CMakeFiles/detector.dir/all' failed
make[1]: *** [CMakeFiles/detector.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
You can see that I do have opencv in place:
stiv@nano:~/yolo-tensorrt/build$ locate opencv2/opencv.hpp
/home/stiv/jsoft/opencv-4.1.2/include/opencv2/opencv.hpp
/home/stiv/jsoft-1.0/opencv-4.1.2/include/opencv2/opencv.hpp
/usr/include/opencv4/opencv2/opencv.hpp
/usr/local/include/opencv4/opencv2/opencv.hpp
can we fix it somehow?
Thank you for your work, but there are some questions. The speed of the three inference precison (INT8 FP16 FP32) for the one model is the same, I only modify the config.inference_precison?
yolov4 runs successfully
but yolov4-tiny has no output. res.size is 0
Config config_v4;
config_v4.net_type = YOLOV4;
config_v4.file_model_cfg = "./model/yolov4-tiny.cfg";
config_v4.file_model_weights = "./model/yolov4-tiny.weights";
config_v4.inference_precison = FP32;
您好，您试过尺寸不是1：1的模型吗，我试尺寸是1：1的测试成功，不是1：1的测试结果为空
Hi,
I was building on Ubuntu 18.04 with OpenCV installed.
The cmake went fine. Then, I started to make and I got this error :
make
Scanning dependencies of target detector
[ 7%] Building CXX object CMakeFiles/detector.dir/modules/calibrator.cpp.o
In file included from /home/anis/TensorRT/yolo-tensorrt/modules/ds_image.h:28:0,
from /home/anis/TensorRT/yolo-tensorrt/modules/calibrator.h:29,
from /home/anis/TensorRT/yolo-tensorrt/modules/calibrator.cpp:26:
/home/anis/TensorRT/yolo-tensorrt/modules/trt_utils.h:34:10: fatal error: opencv2/dnn/dnn.hpp: No such file or directory
#include <opencv2/dnn/dnn.hpp>
^~~~~~~~~~~~~~~~~~~~~
compilation terminated.
CMakeFiles/detector.dir/build.make:243: recipe for target 'CMakeFiles/detector.dir/modules/calibrator.cpp.o' failed
make[2]: *** [CMakeFiles/detector.dir/modules/calibrator.cpp.o] Error 1
CMakeFiles/Makefile2:124: recipe for target 'CMakeFiles/detector.dir/all' failed
make[1]: *** [CMakeFiles/detector.dir/all] Error 2
Makefile:103: recipe for target 'all' failed
make: *** [all] Error 2I rechecked the openCV build and I couldn't find anything missing.
if anyone has feedback on this, please hit me.
您好，还想请教一个问题，AlexeyAB darknet训练基于的格式是RGB格式，opencv默认读出来的格式是BGR格式，请问您的代码中对通道做了转换吗，还有图像归一化使用的mean和std都是多少呀
是不是不支持yolov5的2.0以前的版本？我用的刚开始的版本，可以生成cfg和weight，但是在detector_init(config_v5)会报错
作者你好，我看到yolov4中netAddUpsample函数，使用了addConstant addMatrixMultiply等函数。其背后的原理是什么？ 不是太理解，如何利用这几次操作实现了上采样？
after cmake .. ok, when make, the error happen:
the fatal error: API.h: No such file or directory
is therea someway to find this file?
Here: https://github.com/enazoe/yolo-tensorrt/blob/master/modules/yolo.cpp#L666
This loop calls 3 times and m_ClassNames containts 240 elements: 0, 1, 2, ..., 79, 0, 1, 2, ..., 79, 0, 1, 2, ..., 79
I advance add condition:
if (m_ClassNames.empty())
{
for (int i=0;i< outputTensor.numClasses;++i)
{
m_ClassNames.push_back(std::to_string(i));
}
}
[ 94%] Building CXX object CMakeFiles/yolo-trt.dir/samples/sample_detector.cpp.o
/usr/bin/c++ -I/usr/local/cuda-10.0/include -I/home/jetsonnano/Desktop/yolo-tensorrt/modules -I/home/jetsonnano/Desktop/yolo-tensorrt/extra -isystem /usr/local/include/opencv4 -std=c++14 -Wno-write-strings -std=gnu++11 -o CMakeFiles/yolo-trt.dir/samples/sample_detector.cpp.o -c /home/jetsonnano/Desktop/yolo-tensorrt/samples/sample_detector.cpp
/home/jetsonnano/Desktop/yolo-tensorrt/samples/sample_detector.cpp: In function ‘int main()’:
/home/jetsonnano/Desktop/yolo-tensorrt/samples/sample_detector.cpp:40:45: error: ‘make_unique’ is not a member of ‘std’
std::unique_ptr detector_ = std::make_unique();
^~~~~~~~~~~
/home/jetsonnano/Desktop/yolo-tensorrt/samples/sample_detector.cpp:40:45: note: suggested alternative: ‘_unique’
std::unique_ptr detector = std::make_unique();
^~~~~~~~~~~
unique
/home/jetsonnano/Desktop/yolo-tensorrt/samples/sample_detector.cpp:40:65: error: expected primary-expression before ‘>’ token
std::unique_ptr detector = std::make_unique();
^
/home/jetsonnano/Desktop/yolo-tensorrt/samples/sample_detector.cpp:40:67: error: expected primary-expression before ‘)’ token
std::unique_ptr detector = std::make_unique();
^
CMakeFiles/yolo-trt.dir/build.make:62: recipe for target 'CMakeFiles/yolo-trt.dir/samples/sample_detector.cpp.o' failed
make[2]: *** [CMakeFiles/yolo-trt.dir/samples/sample_detector.cpp.o] Error 1
make[2]: Leaving directory '/home/jetsonnano/Desktop/yolo-tensorrt/build'
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/yolo-trt.dir/all' failed
make[1]: *** [CMakeFiles/yolo-trt.dir/all] Error 2
make[1]: Leaving directory '/home/jetsonnano/Desktop/yolo-tensorrt/build'
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.