monday-leo / yolov7_tensorrt Goto Github PK

我按照您的readme在开发板上进行了模型部署，一切顺利，但是检测物体的置信度为负数，这是因为什么原因造成的，因为我没有在trt推理下评估模型的指标，所以我很难确定这样带来的性能影响，如果你可以回答我的疑问，我将很感激

运行infer.py后出现KeyError:'num_dets'，请问应该怎么解决

can you test the mAP of Yolov7 in coco2017val with TensorRT？ Thank you.

[07/10/2022-18:30:00] [E] [TRT] ModelImporter.cpp:774: --- Begin node ---
[07/10/2022-18:30:00] [E] [TRT] ModelImporter.cpp:775: input: "642"
input: "1341"
output: "645"
name: "Mul_446"
op_type: "Mul"

[07/10/2022-18:30:00] [E] [TRT] ModelImporter.cpp:776: --- End node ---
[07/10/2022-18:30:00] [E] [TRT] ModelImporter.cpp:779: ERROR: ModelImporter.cpp:180 In function parseGraph:
[6] Invalid Node - Mul_446
[graphShapeAnalyzer.cpp::analyzeShapes::1294] Error Code 4: Miscellaneous (IElementWiseLayer Mul_446: broadcast dimensions must be conformable)
[07/10/2022-18:30:00] [E] Failed to parse onnx file
[07/10/2022-18:30:00] [I] Finish parsing network model
[07/10/2022-18:30:00] [E] Parsing model failed
[07/10/2022-18:30:00] [E] Failed to create engine from model or file.
[07/10/2022-18:30:00] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8401] # ./trtexec --onnx=./yolov7.onnx --saveEngine=./yolov7_fp16.engine --fp16 --workspace=200

`D:\Anaconda3\envs\torch17\python.exe D:/work_place/yolov7_coil_test/infer_python.py
[07/18/2022-18:04:34] [TRT] [I] [MemUsageChange] Init CUDA: CPU +578, GPU +0, now: CPU 8750, GPU 961 (MiB)
[07/18/2022-18:04:34] [TRT] [I] Loaded engine size: 75 MiB
[07/18/2022-18:04:34] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.2.0
[07/18/2022-18:04:34] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +701, GPU +262, now: CPU 9535, GPU 1295 (MiB)
[07/18/2022-18:04:35] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +476, GPU +254, now: CPU 10011, GPU 1549 (MiB)
[07/18/2022-18:04:35] [TRT] [W] TensorRT was linked against cuDNN 8.2.1 but loaded cuDNN 8.0.4
[07/18/2022-18:04:35] [TRT] [I] [MemUsageChange] TensorRT-managed allocation in engine deserialization: CPU +0, GPU +71, now: CPU 0, GPU 71 (MiB)

Process finished with exit code -1073741819 (0xC0000005)
`
我的转出来的onnx是：

使用export_onnx.py 文件生成了onnx文件，在使用trtexec 生成engine 文件时失败，环境为 cuda10.2 cudnn8.4.1 torch1.7.1 是否在生成onnx文件时需要将device 设置为GPU模式

相关环境：
python 3.6.9
cuda-10.2
cuDNN-8.4.1
TensorRT-8.4.3.1
转的是官方的yolov7.pt，报错如下，搜了下有人说是cuda-10.2的问题，需要装2个补丁，我装过了还是报这个错，求助下

[10/12/2022-11:13:58] [W] [TRT] onnx2trt_utils.cpp:369: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[10/12/2022-11:13:58] [W] [TRT] onnx2trt_utils.cpp:395: One or more weights outside the range of INT32 was clamped
[10/12/2022-11:13:58] [I] [TRT] No importer registered for op: EfficientNMS_TRT. Attempting to import as plugin.
[10/12/2022-11:13:58] [I] [TRT] Searching for plugin: EfficientNMS_TRT, plugin_version: 1, plugin_namespace: 
[10/12/2022-11:13:58] [I] [TRT] Successfully created plugin: EfficientNMS_TRT
[10/12/2022-11:13:58] [I] Finish parsing network model
[10/12/2022-11:15:26] [I] [TRT] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +101, GPU +50, now: CPU 450, GPU 207 (MiB)
[10/12/2022-11:16:56] [I] [TRT] [MemUsageChange] Init cuDNN: CPU +124, GPU +52, now: CPU 574, GPU 259 (MiB)
[10/12/2022-11:16:56] [I] [TRT] Local timing cache in use. Profiling results in this builder pass will not be stored.
[10/12/2022-11:20:42] [I] [TRT] Some tactics do not have sufficient workspace memory to run. Increasing workspace size will enable more tactics, please check verbose output for requested sizes.
[10/12/2022-11:21:06] [E] Error[2]: [ltWrapper.cpp::setupHeuristic::349] Error Code 2: Internal Error (Assertion cublasStatus == CUBLAS_STATUS_SUCCESS failed. )
[10/12/2022-11:21:06] [E] Error[2]: [builder.cpp::buildSerializedNetwork::636] Error Code 2: Internal Error (Assertion engine != nullptr failed. )
[10/12/2022-11:21:06] [E] Engine could not be created from network
[10/12/2022-11:21:06] [E] Building engine failed
[10/12/2022-11:21:06] [E] Failed to create engine from model or file.
[10/12/2022-11:21:06] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8403] # ./trtexec --onnx=./yolov7.onnx --saveEngine=./yolov7.engine --workspace=200

博主您好，看了您的教程在用tensorrt生成engine模型时正常，但是在infer时会报错，想问下是什么原因？
另外我在用官方的export时导出onnx模型再生成engine时不会报这个错，但是infer出来的图片预测框全堆在一起，想问下是什么原因。

能否出一版C++的推理呢

已生成出yolov7_fp16.engine,但出現如下錯誤

Traceback (most recent call last):
File "/jupyter/local/Github/sreaming/tensorrt/infer.py", line 113, in
trt_engine = TRT_engine("./yolov7_fp16.engine")
File "/jupyter/local/Github/sreaming/tensorrt/infer.py", line 13, in init
self.init_engine()
File "/jupyter/local/Github/sreaming/tensorrt/infer.py", line 25, in init_engine
for index in range(self.model.num_bindings):
AttributeError: 'NoneType' object has no attribute 'num_bindings'