open-mmlab / mmdeploy Goto Github PK
View Code? Open in Web Editor NEWOpenMMLab Model Deployment Framework
Home Page: https://mmdeploy.readthedocs.io/en/latest/
License: Apache License 2.0
mmdeploy's People
Contributors
Stargazers
Watchers
Forkers
pppppm vvsssssk hanrui1sensetime grimoire hhaandroid zcmax runningleon allentdan xjtuchenchao triple-mu henryhan960829 chenyiminch canghaiyunfan hyaihjq zhouzaida davis-love-ai cuhk-hbsun singlezombie feigechuanshu zgq91 hellock hwddodo vgg4resnet zhaohb chaomath rankevin sinead-li lvhan028 xiaohan2013 padadox shenmayufei dy-gdut lzhangzz acdart fxwfzsxyq allensmile stephenfang51 humu789 yingfhu mali82 linshuliang nine03 liuqinglong110 mthecomposer gavinljj zhiqwang stormyl devopslernermahe cnliao shenzhangzhou tehkillerbee abchamp wwjwy qzlearning torinchen yangrz7 hxl1990 laplacekorea phatli irexyc taofuyu echo-minn voxuanthuy mzr1996 chaopengz note-liu projektosmium yangdesheng dangyajun harold-lkk www516717402 ypwhs centineo richard-mei semyonbevzuk ccw1996 devops-forked maxpark uniyushu matrixgame2018 waterbearbee irvingzhang0512 haofanwang kayce001 feiyunwill donnymao marenan zacharygao 23119841 bobby20180331 yangxiaoyang1997 zhaoyanjoin af-74413592 livings5858 yumendecc wutongshenqiu wep21 jaminjiang arctanbell x-yy0mmdeploy's Issues
instance segmentation support/example using C API
Describe the feature
Instance segmentation does not appear to be supported by the C API at this point. This would be a very welcome addition.
Motivation
I looked at the MMDeploy SDK C code and noticed that the examples do not cover Instance segmentation.
However, it is possible to deploy a model for instance segmentation (Mask RCNN) using TensorRT backend. The resulting TensorRT model works correctly. Visualizing the output shows identical output for PyTorch/TensorRT (albeit with incorrect CLASSES, since I use custom classes). Running the same model with the "object_detection.cpp" does work, although with no masks.
If this feature is already supported internally by the C API, feel free to point me in the right direction so I can make an example application myself.
C++ example is all i need
reason:
Tensorrt deployment always use C++, and there have some C code in mmdeploy. It is difficult to read directly without examples.
request:
I hope mmdeploy can release an example of running an Engine In C++. In this way, we can better understand the code.
Thanks for mmdeploy's open source :)
ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin TRTBatchedNMS version 1
ile does not exist:
[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: builtin_op_importers.cpp:3771: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin TRTBatchedNMS version 1
2022-01-12:01:59:25,root ERROR [utils.py:41] Failed to parse onnx, In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
Traceback (most recent call last):
File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper
result = target(*args, **kwargs)
File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt
device_id=device_id)
File "/mnt/d/code/program/硕士毕业论文/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 76, in create_trt_engine
raise RuntimeError(f'Failed to parse onnx, {error_msgs}')
RuntimeError: Failed to parse onnx, In node -1 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
2022-01-12 01:59:25,808 - mmdeploy - ERROR - onnx2tensorrt of work_dir/end2end.onnx failed.
Build failed with following the doc
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
-- Found TensorRT headers at /opt/deps/TensorRT-8.0.3.4/include
-- Found TensorRT libs at /opt/deps/TensorRT-8.0.3.4/lib/libnvinfer.so;/opt/deps/TensorRT-8.0.3.4/lib/libnvinfer_plugin.so
-- Found TENSORRT: /opt/deps/TensorRT-8.0.3.4/include
-- spdlog is found
-- Found OpenCV: /usr (found version "3.2.0")
CMake Error at csrc/preprocess/cuda/CMakeLists.txt:10 (find_package):
By not providing "Findpplcv.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "pplcv", but
CMake did not find one.
Could not find a package configuration file provided by "pplcv" with any of
the following names:
pplcvConfig.cmake
pplcv-config.cmake
Add the installation prefix of "pplcv" to CMAKE_PREFIX_PATH or set
"pplcv_DIR" to a directory containing one of the above files. If "pplcv"
provides a separate development package or SDK, be sure it has been
installed.
-- Configuring incomplete, errors occurred!
See also "/opt/mmdeploy/build/CMakeFiles/CMakeOutput.log".
Reproduction
- What command or script did you run?
Build params:
Step 23/23 : RUN mkdir build && cd build && cmake .. -DMMDEPLOY_BUILD_SDK=ON -DCMAKE_CXX_COMPILER=g++-7 -Dpplcv_DIR=/opt/deps/ppl.cv/install/lib/cmake/ppl -DTENSORRT_DIR=/opt/deps/TensorRT-${TENSORRT_VERSION} -DMMDEPLOY_TARGET_DEVICES="cuda;cpu" -DMMDEPLOY_TARGET_BACKENDS=trt -DMMDEPLOY_CODEBASES=all && cmake --build . -- -j4 && cmake --install .
---> Running in 14f7daf447ab
-- CMAKE_INSTALL_PREFIX: /opt/mmdeploy/build/install
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
- Please run
python tools/check_env.pyto collect necessary environment information and paste it here. - You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as
$PATH,$LD_LIBRARY_PATH,$PYTHONPATH, etc.)
master branch.
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
[Bug] Can't get attribute 'target_wrapper' on <module '__main__> of deploy.py in debug mode
Reproduction
In debug mode, deploy will have attribute errors. The reason is that spawn starts multiple processes and does not allow target_wrapper to be located under the __main__ module. You can refer to ipython/ipython#10894
Error traceback
(<class 'AttributeError'>,
AttributeError("Can't get attribute 'target_wrapper' on <module '__main__' (built-in)>"),
<traceback object at 0x7fb63cdb7b40>)
Bug fix
Move the target_wrapper function to other files.
When exporting to TensorRT, number of detections is capped at 200
Hello,
Thanks for this nice toolkit - so far it is working as expected, a part from a few minor issues.
After deploying my mmdetection model using the TensorRT backend, I noticed that the number of detections (bbox_count) is capped at 200, even though max_per_img in my model config has been set to 1000.
It looks like the number of detections is capped by mmdeploy; base_static.py config is used as a base for the tensorrt deploy_cfg.
Increasing the variable max_output_boxes_per_class solves the problem. But naturally, there are other parameters in the model cfg that differ from the ones set in the deploy_cfg. Shouldn't the max_output_boxes_per_class and other relevant variables be set from the model config automatically?
What is the correct approach? According to the documentation, I can use the generic deploy configs included by mmdeploy without modification (eg. configs/mmdet/instance-seg/instance-seg_tensorrt-fp16_dynamic-320x320-1344x1344.py, as I have been using up to this point). Should I make my own deploy_cfg instead?
Error when onnx2trt
| My env | Ubuntu18.04 |
|---|---|
| gcc g++ | 7.5.0 |
| cmake | 3.20.0 |
| cuda | 10.2 |
| cudnn | 8.0.3 |
| tensorrt | 7.2.1.6 |
| python | 3.7.11 |
| pytorch | 1.8.1 |
| torchvison | 0.9.1 |
| onnx | 1.10.2 |
Hi, I am using mmdeploy to convert maskrcnn pth model from mmdet to tensorrt, but i got this error.
python tools/deploy.py \
> configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py \
> /home/Data/mmlab/mmdetection-2.20.0/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py \
> /home/Data/mmlab/download_weights/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth \
> /home/Data/mmlab/mmdetection-2.20.0/demo/demo.jpg \
> --work-dir work_dirs/maskrcnn/ \
> --device cuda:02022-01-24 15:36:04,005 - mmdeploy - INFO - torch2onnx start.
load checkpoint from local path: /home/Data/mmlab/download_weights/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth
/home/Data/mmlab/mmdetection-2.20.0/mmdet/datasets/utils.py:69: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
'data pipeline in your config file.', UserWarning)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
/home/Data/mmlab/mmdetection-2.20.0/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
/home/Data/mmlab/mmdetection-2.20.0/mmdet/core/anchor/anchor_generator.py:333: UserWarning: ``grid_anchors`` would be deprecated soon. Please use ``grid_priors``
warnings.warn('``grid_anchors`` would be deprecated soon. '
/home/Data/mmlab/mmdetection-2.20.0/mmdet/core/anchor/anchor_generator.py:370: UserWarning: ``single_level_grid_anchors`` would be deprecated soon. Please use ``single_level_grid_priors``
'``single_level_grid_anchors`` would be deprecated soon. '
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/models/dense_heads/rpn_head.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/pytorch/functions/topk.py:54: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if k > size:
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(0) == bboxes.size(0)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(1) == bboxes.size(1)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:167: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
out_boxes = min(num_boxes, after_topk)
/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/mmcv/ops/nms.py:181: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
(batch_size, out_boxes)).to(scores.device)
/home/etop/miniconda3/envs/mmdeploy/lib/python3.7/site-packages/torch/onnx/symbolic_opset9.py:2624: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
"If indices include negative values, the exported graph will produce incorrect results.")
2022-01-24 15:36:18,485 - mmdeploy - INFO - torch2onnx success.
2022-01-24 15:36:18,560 - mmdeploy - INFO - onnx2tensorrt of work_dirs/maskrcnn/end2end.onnx start.
2022-01-24 15:36:19,327 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/Data/mmlab/mmdeploy-0.1.0/build/lib/libmmdeploy_tensorrt_ops.so
[TensorRT] WARNING: onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: builtin_op_importers.cpp:3770: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[TensorRT] INFO: builtin_op_importers.cpp:3787: Successfully created plugin: TRTBatchedNMS
[TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: MMCVMultiLevelRoiAlign. Attempting to import as plugin.
[TensorRT] INFO: builtin_op_importers.cpp:3770: Searching for plugin: MMCVMultiLevelRoiAlign, plugin_version: 1, plugin_namespace:
[TensorRT] INFO: builtin_op_importers.cpp:3787: Successfully created plugin: MMCVMultiLevelRoiAlign
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: onnx2trt_utils.cpp:246: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: Tensor DataType is determined at build time for tensors not marked as input or output.
[TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: builtin_op_importers.cpp:3770: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[TensorRT] INFO: builtin_op_importers.cpp:3787: Successfully created plugin: TRTBatchedNMS
[TensorRT] INFO: ModelImporter.cpp:135: No importer registered for op: MMCVMultiLevelRoiAlign. Attempting to import as plugin.
[TensorRT] INFO: builtin_op_importers.cpp:3770: Searching for plugin: MMCVMultiLevelRoiAlign, plugin_version: 1, plugin_namespace:
[TensorRT] INFO: builtin_op_importers.cpp:3787: Successfully created plugin: MMCVMultiLevelRoiAlign
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: Output type must be INT32 for shape outputs
[TensorRT] WARNING: TensorRT was linked against cuDNN 8.0.4 but loaded cuDNN 8.0.3
[TensorRT] INFO: Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[TensorRT] INTERNAL ERROR: Assertion failed: cublasStatus == CUBLAS_STATUS_SUCCESS
../rtSafe/cublas/cublasLtWrapper.cpp:279
Aborting...
[TensorRT] ERROR: ../rtSafe/cublas/cublasLtWrapper.cpp (279) - Assertion Error in getCublasLtHeuristic: 0 (cublasStatus == CUBLAS_STATUS_SUCCESS)
2022-01-24:15:36:54,root ERROR [utils.py:41] Failed to create TensorRT engine
Traceback (most recent call last):
File "/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/utils/utils.py", line 36, in target_wrapper
result = target(*args, **kwargs)
File "/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt
device_id=device_id)
File "/home/Data/mmlab/mmdeploy-0.1.0/mmdeploy/backend/tensorrt/utils.py", line 116, in create_trt_engine
assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-01-24 15:36:54,970 - mmdeploy - ERROR - onnx2tensorrt of work_dirs/maskrcnn/end2end.onnx failed.But I can run convert yolov3 pth model to tensorrt successfully. I'm not sure if it's because I didn't install the patch for cuda10.2 correctly, hope you can give me a reply.
onnx 转tensorrt错误
环境: cuda 11.1 torch1.8 tensorrt8.2.0.6
执行 tools/onnx2tensorrt.py 报错“ImportError: cannot import name 'create_trt_engine' from 'mmdeploy.backend.tensorrt'”
tensorrt已经按文档安装并且在build/lib/libmmdeploy_tensorrt_ops.so文件
'DataContainer' object is not subscriptable during TensorRT
Hello,
Im trying to convert CRNN text recognition model to TensorRT backbone but getting an error. I was able to convert the pytorch model to TensorRT without any issue and also i was able to measure the performance without error. However, when I try to evaluate the model Im facing the following error. Any idea how can I solve the issue??
2022-01-25 11:29:44,658 - mmdeploy - INFO - Sorting the dataset by 'height' and 'width' is not possible.
2022-01-25 11:29:44,783 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/Documents/Dev/MMDeploy/build/lib/libmmdeploy_tensorrt_ops.so
2022-01-25 11:29:44,783 - mmdeploy - INFO - Successfully loaded tensorrt plugins from /home/Documents/Dev/MMDeploy/build/lib/libmmdeploy_tensorrt_ops.so
[01/25/2022-11:29:45] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
[01/25/2022-11:29:46] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.5.1
[ ] 0/7672, elapsed: 0s, ETA:Traceback (most recent call last):
File "./tools/test.py", line 140, in <module>
main()
File "./tools/test.py", line 133, in main
args.show_dir)
File "/home/Documents/Dev/MMDeploy/mmdeploy/codebase/base/task.py", line 138, in single_gpu_test
out_dir, **kwargs)
File "/home/Documents/Dev/MMDeploy/mmdeploy/codebase/mmocr/deploy/mmocr.py", line 141, in single_gpu_test
outputs = single_gpu_test(model, data_loader, show, out_dir, **kwargs)
File "/home/anaconda3/envs/mmdeploy/lib/python3.7/site-packages/mmdet/apis/test.py", line 32, in single_gpu_test
if batch_size == 1 and isinstance(data['img'][0], torch.Tensor):
TypeError: 'DataContainer' object is not subscriptableThis is the command I use to evaluate the performance
DEPLOY_CFG="configs/mmocr/text-recognition/text-recognition_tensorrt-fp16_dynamic-1x32x32-1x32x640.py"
MODEL_CFG="/home/Documents/Dev/mmocr/configs/textrecog/crnn/crnn_academic_dataset.py"
python ./tools/test.py ${DEPLOY_CFG} ${MODEL_CFG} --model "/home/Desktop/ocr-tensorRT/end2end.engine" --out /home/Desktop/ocr-tensorRT/out/out_ocr.pkl --show-dir "/home/Desktop/ocr-tensorRT"
Thanks!
Plan to support caffe?
is there a plan to export mm* to caffemodel?
Make it possible to silence MMDeploy spdlog logger instance, move some log messages to DEBUG
Hello,
I have noticed that mmdeploy SDK generates quite a bit of log output when running inference. Therefore, I wanted to silence it. I did that by setting the log level early in my program.
auto mmdepLogger = mmdeploy::GetLogger();
mmdepLogger->set_level(spdlog::level::off);
While this almost works as expected, it looks like mmdeploy still manages to print output to the log.
[2022-01-14 16:48:58.176] [mmdeploy] [info] [model.cpp:97] Register 'DirectoryModel'
Is there a better way to silence mmdeploy? :)
Maybe some of these log messages should be DEBUG only to avoid them showing up?
cuda11+ trt7 编译example，出现问题？？
问题一.编译SDK的c example demo的时候，发现找不到动态库。指定了config.cmake的路径。
我在cmakelist里面打印
message(STATUS "need ${MMDeploy_LIBS}") 然后cmke .. ${MMDEPLOY_DIR}/build/install/lib/cmake/MMDeploy .. 显示
-- need -Wl,--no-as-needed;-Wl,--as-needed;-Wl,--whole-archive;-Wl,--no-whole-archive
然后对照MMDeployConfig.cmake中
set(MMDeploy_LIBS ${MMDEPLOY_LIBS}
-Wl,--no-as-needed ${MMDEPLOY_DYNAMIC_MODULES} -Wl,--as-needed
-Wl,--whole-archive ${MMDEPLOY_STATIC_MODULES} -Wl,--no-whole-archive)
其中是不是MMDEPLOY_DYNAMIC_MODULES MMDEPLOY_STATIC_MODULES 变量没有赋值的。
Installation incompatibility between `openvino-dev` and requirements in `mmpdeloy`
During install openvino-dev:
numpy<1.20,>=1.16.6 will be installed
20:08:13 Step 12/12 : RUN pip install openvino-dev
20:08:13 ---> Running in a59e1cd35951
20:08:17 Collecting openvino-dev
20:08:17 Downloading openvino_dev-2021.4.2-3976-py3-none-any.whl (6.2 MB)
20:08:19 Collecting shapely>=1.7.1
20:08:19 Downloading Shapely-1.8.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.1 MB)
20:08:19 Requirement already satisfied: addict>=2.4.0 in /usr/local/lib/python3.8/dist-packages (from openvino-dev) (2.4.0)
20:08:19 Collecting fast-ctc-decode>=0.2.5
20:08:19 Downloading fast_ctc_decode-0.3.0-cp38-cp38-manylinux2010_x86_64.whl (508 kB)
20:08:20 Collecting networkx~=2.5
20:08:20 Downloading networkx-2.6.3-py3-none-any.whl (1.9 MB)
20:08:21 Collecting numpy<1.20,>=1.16.6
20:08:21 Downloading numpy-1.19.5-cp38-cp38-manylinux2010_x86_64.whl (14.9 MB)
While there is no limitation on the version of numpy in requiements/requiement_build.txt and requiements/requiement_runtime.txt currently.
TensorRT deploy with batch size > 1
Thanks for this great work!
I am trying to figure out how to have an export with a larger than 1 batch size on DeepLabV3Plus. That's something I was able to do in mmsegmentation with the following two steps (example with a batchsize=6):
python3 tools/pytorch2onnx.py configs/deeplabv3plus/deeplabv3plus_r18-d8_512x1024_80k_cityscapes.py --checkpoint checkpoints/deeplabv3plus_r18-d8_512x1024_80k_cityscapes_20201226_080942-cff257fe.pth --output-file checkpoints/deeplabv3plus_r18-d8_512x1024_80k_cityscapes_20201226_080942-cff257fe.onnx --input-img data/cityscapes/leftImg8bit/val/frankfurt/frankfurt_000001_032711_leftImg8bit.png --dynamic-export --verify --cfg-options model.test_cfg.mode="whole"
python3 tools/onnx2tensorrt.py configs/deeplabv3plus/deeplabv3plus_r18-d8_512x1024_80k_cityscapes.py checkpoints/deeplabv3plus_r18-d8_512x1024_80k_cityscapes_20201226_080942-cff257fe.onnx --trt-file checkpoints/deeplabv3plus_r18-d8_512x1024_80k_cityscapes_20201226_080942-cff257fe.trt --input-img data/cityscapes/leftImg8bit/val/frankfurt/frankfurt_000001_032711_leftImg8bit.png --verify --min-shape 6 3 512 1024 --max-shape 6 3 512 1024 --fp16
Now, in MMDeploy I try to accomplish the same by changing [min|opt|max]_shape in the TensorRT config file. For instance, I set the file MMDeploy/configs/mmseg/segmentation_tensorrt-fp16_dynamic-512x1024-2048x2048.py to:
_base_ = ['./segmentation_dynamic.py', '../_base_/backends/tensorrt-fp16.py']
backend_config = dict(
common_config=dict(max_workspace_size=1 << 30),
model_inputs=[
dict(
input_shapes=dict(
input=dict(
min_shape=[6, 3, 512, 1024],
opt_shape=[6, 3, 1024, 2048],
max_shape=[6, 3, 2048, 2048])))
])
However, this is my trace error:
2022-01-05:08:13:55,root INFO [deploy.py:72] torch2onnx start.
load checkpoint from local path: ../mmsegmentation/pretrain/deeplabv3plus_r18-d8_512x1024_80k_cityscapes_20201226_080942-cff257fe.pth
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /media/nvidia/NVME/pytorch/pytorch-v1.9.0/c10/core/TensorImpl.h:1156.)
return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
/MMDeploy/mmdeploy/codebase/mmseg/models/segmentors/encoder_decoder.py:39: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
shape = [int(_) for _ in shape]
2022-01-05:08:14:18,root INFO [deploy.py:86] torch2onnx success.
2022-01-05:08:14:19,root INFO [deploy.py:72] onnx2tensorrt of work-dirs/mmseg/deeplabv3plus/deeplabv3_r18-d8_512x1024_80k_cityscapes_20201225_021506-23dffbe2_int8.trt/end2end.onnx start.
2022-01-05:08:14:28,root INFO [init_plugins.py:34] Successfully loaded tensorrt plugins from /MMDeploy/build/lib/libmmdeploy_tensorrt_ops.so
[TensorRT] INFO: [MemUsageChange] Init CUDA: CPU +354, GPU +0, now: CPU 434, GPU 12014 (MiB)
[TensorRT] WARNING: onnx2trt_utils.cpp:364: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: Tensor DataType is determined at build time for tensors not marked as input or output.
[TensorRT] WARNING: DLA requests all profiles have same min, max, and opt value. All dla layers are falling back to GPU
[TensorRT] INFO: [MemUsageSnapshot] Builder begin: CPU 528 MiB, GPU 12110 MiB
[TensorRT] INFO: ---------- Layers Running on DLA ----------
[TensorRT] INFO: ---------- Layers Running on GPU ----------
[TensorRT] INFO: [GpuLayer] Conv_6 + Relu_7
[TensorRT] INFO: [GpuLayer] Conv_8 + Relu_9
[TensorRT] INFO: [GpuLayer] Conv_10 + Relu_11
[TensorRT] INFO: [GpuLayer] MaxPool_12
[TensorRT] INFO: [GpuLayer] Conv_13 + Relu_14
[TensorRT] INFO: [GpuLayer] Conv_15 + Add_16 + Relu_17
[TensorRT] INFO: [GpuLayer] Conv_18 + Relu_19
[TensorRT] INFO: [GpuLayer] Conv_20 + Add_21 + Relu_22
[TensorRT] INFO: [GpuLayer] Conv_23 + Relu_24
[TensorRT] INFO: [GpuLayer] Conv_95 + Relu_96
[TensorRT] INFO: [GpuLayer] Conv_25
[TensorRT] INFO: [GpuLayer] Conv_26 + Add_27 + Relu_28
[TensorRT] INFO: [GpuLayer] Conv_29 + Relu_30
[TensorRT] INFO: [GpuLayer] Conv_31 + Add_32 + Relu_33
[TensorRT] INFO: [GpuLayer] Conv_34 + Relu_35
[TensorRT] INFO: [GpuLayer] Conv_36
[TensorRT] INFO: [GpuLayer] Conv_37 + Add_38 + Relu_39
[TensorRT] INFO: [GpuLayer] Conv_40 + Relu_41
[TensorRT] INFO: [GpuLayer] Conv_42 + Add_43 + Relu_44
[TensorRT] INFO: [GpuLayer] Conv_45 + Relu_46
[TensorRT] INFO: [GpuLayer] Conv_47
[TensorRT] INFO: [GpuLayer] Conv_48 + Add_49 + Relu_50
[TensorRT] INFO: [GpuLayer] Conv_51 + Relu_52
[TensorRT] INFO: [GpuLayer] Conv_53 + Add_54 + Relu_55
[TensorRT] INFO: [GpuLayer] GlobalAveragePool_56
[TensorRT] INFO: [GpuLayer] Conv_78 + Relu_79
[TensorRT] INFO: [GpuLayer] Conv_80 + Relu_81
[TensorRT] INFO: [GpuLayer] Conv_84 + Relu_85
[TensorRT] INFO: [GpuLayer] Conv_88 + Relu_89
[TensorRT] INFO: [GpuLayer] Conv_57 + Relu_58
[TensorRT] INFO: [GpuLayer] Conv_82 + Relu_83
[TensorRT] INFO: [GpuLayer] Conv_86 + Relu_87
[TensorRT] INFO: [GpuLayer] Conv_90 + Relu_91
[TensorRT] INFO: [GpuLayer] Resize_77
[TensorRT] INFO: [GpuLayer] 327 copy
[TensorRT] INFO: [GpuLayer] Conv_93 + Relu_94
[TensorRT] INFO: [GpuLayer] Resize_115
[TensorRT] INFO: [GpuLayer] 374 copy
[TensorRT] INFO: [GpuLayer] Conv_117 + Relu_118
[TensorRT] INFO: [GpuLayer] Conv_119 + Relu_120
[TensorRT] INFO: [GpuLayer] Conv_121 + Relu_122
[TensorRT] INFO: [GpuLayer] Conv_123 + Relu_124
[TensorRT] INFO: [GpuLayer] Conv_125
[TensorRT] INFO: [GpuLayer] Resize_138
[TensorRT] INFO: [GpuLayer] Transpose_139 + (Unnamed Layer* 136) [Shuffle]
[TensorRT] INFO: [GpuLayer] Softmax_140
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 138) [Shuffle] + Transpose_141
[TensorRT] INFO: [GpuLayer] ArgMax_142
[TensorRT] INFO: [GpuLayer] (Unnamed Layer* 145) [Shuffle] + Reshape_144
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +227, GPU +232, now: CPU 755, GPU 12342 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +307, GPU +304, now: CPU 1062, GPU 12646 (MiB)
[TensorRT] WARNING: Detected invalid timing cache, setup a local cache instead
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 1061, GPU 12646 (MiB)
[TensorRT] ERROR: 4: [shapeCompiler.cpp::evaluateShapeChecks::822] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: reshape would change volume. IShuffleLayer Reshape_144: reshaping failed for tensor: 405)
2022-01-05:08:14:31,root ERROR [deploy.py:67] Failed to create TensorRT engine
Traceback (most recent call last):
File "/saivvy/mmlab/MMDeploy/tools/deploy.py", line 62, in target_wrapper
result = target(*args, **kwargs)
File "/MMDeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt
device_id=device_id)
File "/MMDeploy/mmdeploy/backend/tensorrt/utils.py", line 115, in create_trt_engine
assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-01-05:08:14:32,root ERROR [deploy.py:83] onnx2tensorrt of work-dirs/mmseg/deeplabv3plus/deeplabv3_r18-d8_512x1024_80k_cityscapes_20201225_021506-23dffbe2_int8.trt/end2end.onnx failed.
Am I missing something?
Thanks!
[ut] Ut failed with openvino backend
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
[2022-01-13T02:32:51.455Z] ____________________ test_psphead_forward[Backend.OPENVINO] ____________________
[2022-01-13T02:32:51.455Z]
[2022-01-13T02:32:51.455Z] backend = <Backend.OPENVINO: 'openvino'>
[2022-01-13T02:32:51.455Z]
[2022-01-13T02:32:51.455Z] @pytest.mark.parametrize('backend',
[2022-01-13T02:32:51.455Z] [Backend.ONNXRUNTIME, Backend.OPENVINO, Backend.NCNN])
[2022-01-13T02:32:51.455Z] def test_psphead_forward(backend):
[2022-01-13T02:32:51.455Z] check_backend(backend)
[2022-01-13T02:32:51.455Z] from mmseg.models.decode_heads import PSPHead
[2022-01-13T02:32:51.456Z] head = PSPHead(in_channels=32, channels=16, num_classes=19).eval()
[2022-01-13T02:32:51.456Z]
[2022-01-13T02:32:51.456Z] deploy_cfg = mmcv.Config(
[2022-01-13T02:32:51.456Z] dict(
[2022-01-13T02:32:51.456Z] backend_config=dict(type=backend.value),
[2022-01-13T02:32:51.456Z] onnx_config=dict(output_names=['result'], input_shape=None),
[2022-01-13T02:32:51.456Z] codebase_config=dict(type='mmseg', task='Segmentation')))
[2022-01-13T02:32:51.456Z] inputs = [torch.randn(1, 32, 45, 45)]
[2022-01-13T02:32:51.456Z] model_inputs = {'inputs': inputs}
[2022-01-13T02:32:51.456Z] with torch.no_grad():
[2022-01-13T02:32:51.456Z] model_outputs = get_model_outputs(head, 'forward', model_inputs)
[2022-01-13T02:32:51.456Z] wrapped_model = WrapModel(head, 'forward')
[2022-01-13T02:32:51.456Z] rewrite_inputs = {'inputs': inputs}
[2022-01-13T02:32:51.456Z] rewrite_outputs, is_backend_output = get_rewrite_outputs(
[2022-01-13T02:32:51.456Z] wrapped_model=wrapped_model,
[2022-01-13T02:32:51.456Z] model_inputs=rewrite_inputs,
[2022-01-13T02:32:51.456Z] > deploy_cfg=deploy_cfg)
[2022-01-13T02:32:51.456Z]
[2022-01-13T02:32:51.456Z] tests/test_codebase/test_mmseg/test_mmseg_models.py:229:
[2022-01-13T02:32:51.456Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2022-01-13T02:32:51.456Z] mmdeploy/utils/test.py:519: in get_rewrite_outputs
[2022-01-13T02:32:51.456Z] deploy_cfg)
[2022-01-13T02:32:51.456Z] mmdeploy/utils/test.py:482: in get_backend_outputs
[2022-01-13T02:32:51.456Z] device, output_names)
[2022-01-13T02:32:51.456Z] mmdeploy/codebase/base/backend_model.py:76: in _build_wrapper
[2022-01-13T02:32:51.456Z] ir_model_file=backend_files[0], output_names=output_names)
[2022-01-13T02:32:51.456Z] mmdeploy/backend/openvino/wrapper.py:39: in __init__
[2022-01-13T02:32:51.456Z] from openvino.inference_engine import IECore
[2022-01-13T02:32:51.456Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
[2022-01-13T02:32:51.456Z]
[2022-01-13T02:32:51.456Z] import os
[2022-01-13T02:32:51.456Z] import sys
[2022-01-13T02:32:51.456Z]
[2022-01-13T02:32:51.456Z] if sys.platform == 'win32':
[2022-01-13T02:32:51.456Z] # Installer, yum, pip installs openvino dlls to the different directories
[2022-01-13T02:32:51.456Z] # and those paths need to be visible to the openvino modules
[2022-01-13T02:32:51.456Z] #
[2022-01-13T02:32:51.456Z] # If you're using a custom installation of openvino,
[2022-01-13T02:32:51.456Z] # add the location of openvino dlls to your system PATH.
[2022-01-13T02:32:51.456Z] #
[2022-01-13T02:32:51.456Z] # looking for the libs in the pip installation path by default.
[2022-01-13T02:32:51.456Z] openvino_libs = [os.path.join(os.path.dirname(__file__), '..', '..', 'openvino', 'libs')]
[2022-01-13T02:32:51.456Z] # setupvars.bat script set all libs paths to OPENVINO_LIB_PATHS environment variable.
[2022-01-13T02:32:51.456Z] openvino_libs_installer = os.getenv('OPENVINO_LIB_PATHS')
[2022-01-13T02:32:51.456Z] if openvino_libs_installer:
[2022-01-13T02:32:51.456Z] openvino_libs.extend(openvino_libs_installer.split(';'))
[2022-01-13T02:32:51.456Z] for lib in openvino_libs:
[2022-01-13T02:32:51.456Z] lib_path = os.path.join(os.path.dirname(__file__), lib)
[2022-01-13T02:32:51.456Z] if os.path.isdir(lib_path):
[2022-01-13T02:32:51.456Z] # On Windows, with Python >= 3.8, DLLs are no longer imported from the PATH.
[2022-01-13T02:32:51.456Z] if (3, 8) <= sys.version_info:
[2022-01-13T02:32:51.456Z] os.add_dll_directory(os.path.abspath(lib_path))
[2022-01-13T02:32:51.456Z] else:
[2022-01-13T02:32:51.456Z] os.environ['PATH'] = os.path.abspath(lib_path) + ';' + os.environ['PATH']
[2022-01-13T02:32:51.456Z]
[2022-01-13T02:32:51.456Z] > from .ie_api import *
[2022-01-13T02:32:51.456Z] E ImportError: libpython3.7m.so.1.0: cannot open shared object file: No such file or directory
[2022-01-13T02:32:51.456Z]
[2022-01-13T02:32:51.456Z] ../conda/lib/python3.7/site-packages/openvino/inference_engine/__init__.py:30: ImportError
Reproduction
- What command or script did you run?
A placeholder for the command.
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
[2022-01-13T02:18:21.531Z] 2022-01-13 10:18:21,435 - mmdeploy - INFO - TorchVision: 0.10.0
[2022-01-13T02:18:21.531Z] 2022-01-13 10:18:21,435 - mmdeploy - INFO - OpenCV: 4.5.4
[2022-01-13T02:18:21.531Z] 2022-01-13 10:18:21,435 - mmdeploy - INFO - MMCV: 1.4.0
[2022-01-13T02:18:21.531Z] 2022-01-13 10:18:21,435 - mmdeploy - INFO - MMCV Compiler: GCC 7.5
[2022-01-13T02:18:21.531Z] 2022-01-13 10:18:21,435 - mmdeploy - INFO - MMCV CUDA Compiler: 11.1
[2022-01-13T02:18:21.531Z] 2022-01-13 10:18:21,435 - mmdeploy - INFO - MMDeployment: 0.1.0+76f96f0
[2022-01-13T02:18:22.109Z] 2022-01-13 10:18:21,804 - mmdeploy - INFO - onnxruntime: 1.10.0 ops_is_avaliable : True
[2022-01-13T02:18:22.109Z] 2022-01-13 10:18:21,916 - mmdeploy - INFO - tensorrt: 8.0.3.4 ops_is_avaliable : False
[2022-01-13T02:18:22.109Z] 2022-01-13 10:18:21,920 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False
[2022-01-13T02:18:22.109Z] 2022-01-13 10:18:21,922 - mmdeploy - INFO - pplnn_is_avaliable: False
[2022-01-13T02:18:22.109Z] 2022-01-13 10:18:21,924 - mmdeploy - INFO - openvino_is_avaliable: True
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
Typo error in requirement_tests.txt
mmdeploy/requirements/optional.txt
Line 3 in cfb9099
Bug using ORTwrapper
Here is the code using ortwrapper to inference on device
The model is maskrcnn trained through mmdet
Env:
torch1.8
cuda 11.1
onnx 1.8
from mmdeploy.backend.onnxruntime import ORTWrapper
import torch
onnx_file = 'end2end.onnx'
#gpu
model = ORTWrapper(onnx_file, 'cuda:3', ['dets', 'labels', 'masks'])
inputs = dict(input=torch.randn(1, 3, 768, 1344, device='cuda:3'))
outputs = model(inputs)
print(outputs)
no matter on host or device, it still got the error on reshape1588 node
[[21~2021-12-31 13:20:33.759103638 [E:onnxruntime:, sequential_executor.cc:339 Execute] Non-zero status code returned while running Reshape node. Name:'Reshape_1588' Status Message: /onnxruntime_src/onnxruntime/core/providers/cpu/tensor/reshape_helper.h:42 onnxruntime::ReshapeHelper::ReshapeHelper(const onnxruntime::TensorShape&, std::vector&, bool) gsl::narrow_cast<int64_t>(input_shape.Size()) == size was false. The input tensor cannot be reshaped to the requested shape. Input shape:{0,3}, requested shape:{1,3,3}
but works fine using ./tools/test.py, did not see differnece between using ORTwrapper directly.
[Question] Why not use get_logger in MMCV, but directly use the original logging
Most OpenMMLab open source libraries directly reuse mmcv.get_logger, but why mmdeploy uses logging directly, is there any special reason? I think this approach is inconsistent with the style of OpenMMLab.
Unittest run failed with `numpy.ndarray size changed, may indicate binary incompatibility.`
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
A clear and concise description of what the bug is.
Reproduction
- What command or script did you run?
Usingonnxruntimeas the backend
Numpyversion:
numpy 1.22.1
[2022-01-19T11:27:25.331Z] ==================================== ERRORS ====================================
[2022-01-19T11:27:25.331Z] ______ ERROR collecting tests/test_codebase/test_mmdet/test_mmdet_core.py ______
[2022-01-19T11:27:25.331Z] tests/test_codebase/test_mmdet/test_mmdet_core.py:12: in <module>
[2022-01-19T11:27:25.331Z] import_codebase(Codebase.MMDET)
[2022-01-19T11:27:25.331Z] mmdeploy/codebase/__init__.py:30: in import_codebase
[2022-01-19T11:27:25.331Z] importlib.import_module(f'mmdeploy.codebase.{lib}')
[2022-01-19T11:27:25.331Z] /usr/lib/python3.8/importlib/__init__.py:127: in import_module
[2022-01-19T11:27:25.331Z] return _bootstrap._gcd_import(name[level:], package, level)
[2022-01-19T11:27:25.331Z] <frozen importlib._bootstrap>:1014: in _gcd_import
[2022-01-19T11:27:25.331Z] ???
[2022-01-19T11:27:25.331Z] <frozen importlib._bootstrap>:991: in _find_and_load
[2022-01-19T11:27:25.331Z] ???
[2022-01-19T11:27:25.331Z] <frozen importlib._bootstrap>:975: in _find_and_load_unlocked
[2022-01-19T11:27:25.331Z] ???
[2022-01-19T11:27:25.331Z] <frozen importlib._bootstrap>:671: in _load_unlocked
[2022-01-19T11:27:25.331Z] ???
[2022-01-19T11:27:25.331Z] <frozen importlib._bootstrap_external>:843: in exec_module
[2022-01-19T11:27:25.331Z] ???
[2022-01-19T11:27:25.331Z] <frozen importlib._bootstrap>:219: in _call_with_frames_removed
[2022-01-19T11:27:25.331Z] ???
[2022-01-19T11:27:25.331Z] mmdeploy/codebase/mmdet/__init__.py:5: in <module>
[2022-01-19T11:27:25.331Z] from .models import * # noqa: F401,F403
[2022-01-19T11:27:25.331Z] mmdeploy/codebase/mmdet/models/__init__.py:3: in <module>
[2022-01-19T11:27:25.331Z] from .dense_heads import * # noqa: F401,F403
[2022-01-19T11:27:25.331Z] mmdeploy/codebase/mmdet/models/dense_heads/__init__.py:2: in <module>
[2022-01-19T11:27:25.331Z] from .base_dense_head import (base_dense_head__get_bbox,
[2022-01-19T11:27:25.331Z] mmdeploy/codebase/mmdet/models/dense_heads/base_dense_head.py:2: in <module>
[2022-01-19T11:27:25.331Z] from mmdet.core.bbox.coder import (DeltaXYWHBBoxCoder, DistancePointBBoxCoder,
[2022-01-19T11:27:25.331Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/__init__.py:6: in <module>
[2022-01-19T11:27:25.331Z] from .hook import * # noqa: F401, F403
[2022-01-19T11:27:25.331Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/hook/__init__.py:5: in <module>
[2022-01-19T11:27:25.331Z] from .sync_norm_hook import SyncNormHook
[2022-01-19T11:27:25.331Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/hook/sync_norm_hook.py:8: in <module>
[2022-01-19T11:27:25.332Z] from ..utils.dist_utils import all_reduce_dict
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/utils/__init__.py:4: in <module>
[2022-01-19T11:27:25.332Z] from .misc import (center_of_mass, filter_scores_and_topk, flip_tensor,
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/utils/misc.py:8: in <module>
[2022-01-19T11:27:25.332Z] from ..mask.structures import BitmapMasks, PolygonMasks
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/mask/__init__.py:3: in <module>
[2022-01-19T11:27:25.332Z] from .structures import BaseInstanceMasks, BitmapMasks, PolygonMasks
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/mask/structures.py:7: in <module>
[2022-01-19T11:27:25.332Z] import pycocotools.mask as maskUtils
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/pycocotools/mask.py:3: in <module>
[2022-01-19T11:27:25.332Z] import pycocotools._mask as _mask
[2022-01-19T11:27:25.332Z] pycocotools/_mask.pyx:1: in init pycocotools._mask
[2022-01-19T11:27:25.332Z] ???
[2022-01-19T11:27:25.332Z] E ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
[2022-01-19T11:27:25.332Z] _ ERROR collecting tests/test_codebase/test_mmdet/test_object_detection_model.py _
[2022-01-19T11:27:25.332Z] tests/test_codebase/test_mmdet/test_object_detection_model.py:13: in <module>
[2022-01-19T11:27:25.332Z] from mmdeploy.codebase.mmdet.deploy.object_detection_model import End2EndModel
[2022-01-19T11:27:25.332Z] mmdeploy/codebase/mmdet/deploy/object_detection_model.py:10: in <module>
[2022-01-19T11:27:25.332Z] from mmdet.core import bbox2result
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/__init__.py:7: in <module>
[2022-01-19T11:27:25.332Z] from .mask import * # noqa: F401, F403
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/mask/__init__.py:3: in <module>
[2022-01-19T11:27:25.332Z] from .structures import BaseInstanceMasks, BitmapMasks, PolygonMasks
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/mask/structures.py:7: in <module>
[2022-01-19T11:27:25.332Z] import pycocotools.mask as maskUtils
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/pycocotools/mask.py:3: in <module>
[2022-01-19T11:27:25.332Z] import pycocotools._mask as _mask
[2022-01-19T11:27:25.332Z] pycocotools/_mask.pyx:1: in init pycocotools._mask
[2022-01-19T11:27:25.332Z] ???
[2022-01-19T11:27:25.332Z] E ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
[2022-01-19T11:27:25.332Z] _____ ERROR collecting tests/test_codebase/test_mmocr/test_mmocr_models.py _____
[2022-01-19T11:27:25.332Z] tests/test_codebase/test_mmocr/test_mmocr_models.py:8: in <module>
[2022-01-19T11:27:25.332Z] from mmocr.models.textdet.necks import FPNC
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmocr/models/__init__.py:2: in <module>
[2022-01-19T11:27:25.332Z] from . import common, kie, textdet, textrecog
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmocr/models/common/__init__.py:2: in <module>
[2022-01-19T11:27:25.332Z] from . import backbones, losses
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmocr/models/common/backbones/__init__.py:2: in <module>
[2022-01-19T11:27:25.332Z] from .unet import UNet
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmocr/models/common/backbones/unet.py:9: in <module>
[2022-01-19T11:27:25.332Z] from mmocr.models.builder import (BACKBONES, UPSAMPLE_LAYERS,
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmocr/models/builder.py:8: in <module>
[2022-01-19T11:27:25.332Z] from mmdet.models.builder import BACKBONES as MMDET_BACKBONES
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/models/__init__.py:7: in <module>
[2022-01-19T11:27:25.332Z] from .dense_heads import * # noqa: F401,F403
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/models/dense_heads/__init__.py:2: in <module>
[2022-01-19T11:27:25.332Z] from .anchor_free_head import AnchorFreeHead
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/models/dense_heads/anchor_free_head.py:10: in <module>
[2022-01-19T11:27:25.332Z] from mmdet.core import build_bbox_coder, multi_apply
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/__init__.py:7: in <module>
[2022-01-19T11:27:25.332Z] from .mask import * # noqa: F401, F403
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/mask/__init__.py:3: in <module>
[2022-01-19T11:27:25.332Z] from .structures import BaseInstanceMasks, BitmapMasks, PolygonMasks
[2022-01-19T11:27:25.332Z] /usr/local/lib/python3.8/dist-packages/mmdet/core/mask/structures.py:7: in <module>
[2022-01-19T11:27:25.333Z] import pycocotools.mask as maskUtils
[2022-01-19T11:27:25.333Z] /usr/local/lib/python3.8/dist-packages/pycocotools/mask.py:3: in <module>
[2022-01-19T11:27:25.333Z] import pycocotools._mask as _mask
[2022-01-19T11:27:25.333Z] pycocotools/_mask.pyx:1: in init pycocotools._mask
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] E ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
[2022-01-19T11:27:25.333Z] ____ ERROR collecting tests/test_codebase/test_mmocr/test_text_detection.py ____
[2022-01-19T11:27:25.333Z] tests/test_codebase/test_mmocr/test_text_detection.py:17: in <module>
[2022-01-19T11:27:25.333Z] import_codebase(Codebase.MMOCR)
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/__init__.py:30: in import_codebase
[2022-01-19T11:27:25.333Z] importlib.import_module(f'mmdeploy.codebase.{lib}')
[2022-01-19T11:27:25.333Z] /usr/lib/python3.8/importlib/__init__.py:127: in import_module
[2022-01-19T11:27:25.333Z] return _bootstrap._gcd_import(name[level:], package, level)
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:1014: in _gcd_import
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:991: in _find_and_load
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:975: in _find_and_load_unlocked
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:671: in _load_unlocked
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap_external>:843: in exec_module
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:219: in _call_with_frames_removed
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/mmocr/__init__.py:2: in <module>
[2022-01-19T11:27:25.333Z] from .deploy import * # noqa: F401,F403
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/mmocr/deploy/__init__.py:3: in <module>
[2022-01-19T11:27:25.333Z] from .text_detection import TextDetection
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/mmocr/deploy/text_detection.py:8: in <module>
[2022-01-19T11:27:25.333Z] from mmdet.datasets import replace_ImageToTensor
[2022-01-19T11:27:25.333Z] /usr/local/lib/python3.8/dist-packages/mmdet/datasets/__init__.py:3: in <module>
[2022-01-19T11:27:25.333Z] from .cityscapes import CityscapesDataset
[2022-01-19T11:27:25.333Z] /usr/local/lib/python3.8/dist-packages/mmdet/datasets/cityscapes.py:13: in <module>
[2022-01-19T11:27:25.333Z] import pycocotools.mask as maskUtils
[2022-01-19T11:27:25.333Z] /usr/local/lib/python3.8/dist-packages/pycocotools/mask.py:3: in <module>
[2022-01-19T11:27:25.333Z] import pycocotools._mask as _mask
[2022-01-19T11:27:25.333Z] pycocotools/_mask.pyx:1: in init pycocotools._mask
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] E ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
[2022-01-19T11:27:25.333Z] _ ERROR collecting tests/test_codebase/test_mmocr/test_text_detection_models.py _
[2022-01-19T11:27:25.333Z] tests/test_codebase/test_mmocr/test_text_detection_models.py:15: in <module>
[2022-01-19T11:27:25.333Z] import_codebase(Codebase.MMOCR)
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/__init__.py:30: in import_codebase
[2022-01-19T11:27:25.333Z] importlib.import_module(f'mmdeploy.codebase.{lib}')
[2022-01-19T11:27:25.333Z] /usr/lib/python3.8/importlib/__init__.py:127: in import_module
[2022-01-19T11:27:25.333Z] return _bootstrap._gcd_import(name[level:], package, level)
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:1014: in _gcd_import
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:991: in _find_and_load
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:975: in _find_and_load_unlocked
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:671: in _load_unlocked
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap_external>:843: in exec_module
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] <frozen importlib._bootstrap>:219: in _call_with_frames_removed
[2022-01-19T11:27:25.333Z] ???
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/mmocr/__init__.py:2: in <module>
[2022-01-19T11:27:25.333Z] from .deploy import * # noqa: F401,F403
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/mmocr/deploy/__init__.py:3: in <module>
[2022-01-19T11:27:25.333Z] from .text_detection import TextDetection
[2022-01-19T11:27:25.333Z] mmdeploy/codebase/mmocr/deploy/text_detection.py:8: in <module>
[2022-01-19T11:27:25.333Z] from mmdet.datasets import replace_ImageToTensor
[2022-01-19T11:27:25.334Z] /usr/local/lib/python3.8/dist-packages/mmdet/datasets/__init__.py:3: in <module>
[2022-01-19T11:27:25.334Z] from .cityscapes import CityscapesDataset
[2022-01-19T11:27:25.334Z] /usr/local/lib/python3.8/dist-packages/mmdet/datasets/cityscapes.py:13: in <module>
[2022-01-19T11:27:25.334Z] import pycocotools.mask as maskUtils
[2022-01-19T11:27:25.334Z] /usr/local/lib/python3.8/dist-packages/pycocotools/mask.py:3: in <module>
[2022-01-19T11:27:25.334Z] import pycocotools._mask as _mask
[2022-01-19T11:27:25.334Z] pycocotools/_mask.pyx:1: in init pycocotools._mask
[2022-01-19T11:27:25.334Z] ???
[2022-01-19T11:27:25.334Z] E ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,737 - mmdeploy - INFO - sys.platform: linux
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,739 - mmdeploy - INFO - Python: 3.8.12 (default, Jan 15 2022, 18:39:47) [GCC 7.5.0]
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,739 - mmdeploy - INFO - CUDA available: False
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,739 - mmdeploy - INFO - GCC: gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,740 - mmdeploy - INFO - PyTorch: 1.9.0+cpu
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,740 - mmdeploy - INFO - PyTorch compiling details: PyTorch built with:
[2022-01-19T11:23:16.665Z] - GCC 7.3
[2022-01-19T11:23:16.665Z] - C++ Version: 201402
[2022-01-19T11:23:16.665Z] - Intel(R) Math Kernel Library Version 2020.0.0 Product Build 20191122 for Intel(R) 64 architecture applications
[2022-01-19T11:23:16.665Z] - Intel(R) MKL-DNN v2.1.2 (Git Hash 98be7e8afa711dc9b66c8ff3504129cb82013cdb)
[2022-01-19T11:23:16.665Z] - OpenMP 201511 (a.k.a. OpenMP 4.5)
[2022-01-19T11:23:16.665Z] - NNPACK is enabled
[2022-01-19T11:23:16.665Z] - CPU capability usage: AVX2
[2022-01-19T11:23:16.665Z] - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CXX_COMPILER=/opt/rh/devtoolset-7/root/usr/bin/c++, CXX_FLAGS= -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -fopenmp -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wno-narrowing -Wall -Wextra -Werror=return-type -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-unused-local-typedefs -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.9.0, USE_CUDA=0, USE_CUDNN=OFF, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=ON, USE_OPENMP=ON,
[2022-01-19T11:23:16.665Z]
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,740 - mmdeploy - INFO - TorchVision: 0.10.0+rocm4.2
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,740 - mmdeploy - INFO - OpenCV: 4.5.5
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,740 - mmdeploy - INFO - MMCV: 1.4.0
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,741 - mmdeploy - INFO - MMCV Compiler: GCC 7.3
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,741 - mmdeploy - INFO - MMCV CUDA Compiler: not available
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:15,741 - mmdeploy - INFO - MMDeployment: 0.1.0+cfb9099
[2022-01-19T11:23:16.665Z] [2022-01-19 19:23:16.265] [mmdeploy] [info] Register 'DirectoryModel'
[2022-01-19T11:23:16.665Z] 2022-01-19 19:23:16,271 - mmdeploy - INFO - onnxruntime: 1.8.1 ops_is_avaliable : True
[2022-01-19T11:23:16.666Z] 2022-01-19 19:23:16,274 - mmdeploy - INFO - tensorrt: None ops_is_avaliable : False
[2022-01-19T11:23:16.666Z] 2022-01-19 19:23:16,277 - mmdeploy - INFO - ncnn: None ops_is_avaliable : False
[2022-01-19T11:23:16.666Z] 2022-01-19 19:23:16,279 - mmdeploy - INFO - pplnn_is_avaliable: False
[2022-01-19T11:23:16.666Z] 2022-01-19 19:23:16,281 - mmdeploy - INFO - openvino_is_avaliable: False
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
[Question] MMCodebase does not need to be instantiated
def __build_codebase_class(codebase: Codebase, registry: Registry):
return registry.module_dict[codebase.value]
CODEBASE = Registry('Codebases', build_func=__build_codebase_class)I found that CODEBASE-related classes do not need to be instantiated, is this expected behavior? If classes such as MMClassification and MMDetection do not need to be instantiated, it is better to have a clearer description. And the build_dataloader method of MMClassification lacks the @staticmethod decorator. Maybe it's a bug.
def build_dataloader(dataset: Dataset,
samples_per_gpu: int,
workers_per_gpu: int,
num_gpus: int = 1,
dist: bool = False,
shuffle: bool = False,
round_up: bool = True,
seed: Optional[int] = None,
pin_memory: bool = True,
persistent_workers: bool = True,
**kwargs) -> DataLoader:RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
Hi, I am using mmdeploy to convert maskrcnn pth model to tensorrt, and i got this error:
RuntimeError: cuDNN error: CUDNN_STATUS_NOT_INITIALIZED
my command:
python tools/deploy.py \ configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py \ /home/mmlab/mmdetection-2.20.0/project/mrcnn_r18_fpn_160e/mrcnn_r18_fpn_160e.py \ /home/mmlab/mmdetection-2.20.0/project/mrcnn_r18_fpn_160e/epoch_150.pth \ /home/mmlab/mmdetection-2.20.0/demo/demo.jpg \ --work-dir work_dir \ --device cuda:0
my env:
gcc g++ 7.5.0
cuda 10.2
cudnn 7.6.5
tensorrt 7.2.3
torch 1.8.0
[doc] Add requirement for mmcv version in docs
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
We added the requirements in https://mmdeploy.readthedocs.io/en/latest/build.html#create-environment but I can not find the limitation for the mmcv version, maybe we can add it if we can, to avoid to use the latest mmcv.
Reproduction
- What command or script did you run?
A placeholder for the command.
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
- Please run
python tools/check_env.pyto collect necessary environment information and paste it here. - You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as
$PATH,$LD_LIBRARY_PATH,$PYTHONPATH, etc.)
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
Deploying ncnn single stage detection model failed
For retinanet. Failed with configs/mmdet/detection/single-stage_partition_ncnn_static.py. Succeed with configs/mmdet/detection/single-stage_ncnn_static-800x1344.py
Retardation of model inference
I'm inference on the CPU. YOLOF model is converted to ONNX model from 2S to 4S. The YOLOX model is also slowed down. Why is that?
I'm running an error in docker .
I used the conversion command "python /home/zhaoyuan/MMDeploy//tools/deploy.py /home/zhaoyuan/MMDeploy/configs/mmdet/instance-seg/instance-seg_onnxruntime_dynamic.py mask_rcnn_x101_64x4d_fpn_1x_coco.py epoch_100.pth /home/zhaoyuan/code/index_recognize/test/202112221918072815.jpg --work-dir ./ --device cuda:0 --dump-info" to generate the onnx file.
I call using the command "seg_result = inference_model(seg_config_file, "/home/zhaoyuan/MMDeploy/configs/mmdet/instance-seg/instance-seg_onnxruntime_dynamic.py",
["/home/zhaoyuan/code/index_recognize/mask_rcnn_x101_1x/end2end.onnx"], img=bbox_image[0], device=device)[0][1]" in the code.
Then I ran an error in docker:"onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Load model from /root/mm_index/config/end2end.onnx failed:Fatal error: MMCVRoiAlign is not a registered function/op"
why is this ?
There is a bug while visulizing the result.
This function only return the class name for open datasets like Cocodataset, not for custom.
Therefore it can't correctly put the right classname on imgs.
for now I can just change the return value to list(model_cfg.classes) to solve temporary
Thanks !
MMDeploy test.py slower than MMSegmentation deploy_test.py using TensorRT?
Hello, I was using MMDeploy to convert the fastscnn pytorch into a TensorRT engine, which works very well. Testing the TRT model on the MMDeploy tools/test.py results in 10 tasks/second, while testing it on the MMSegmentation tools/deploy_test.py results in 20 tasks/second. Is there reason why the MMDeploy runs slower? I am assuming it is because it was built for a more general case?
Here are the commands I did:
MMDeploy
python tools/test.py configs/mmseg/segmentation_tensorrt_static-1024x2048.py /home/calvinnguyenq/dev/MMSegmentation/mmsegmentation/configs/fastscnn/fast_scnn_lr0.12_8x4_160k_cityscapes.py --model demo/end2end.engine --speed-test --device cuda:0
MMSegmentation
python tools/deploy_test.py configs/fastscnn/fast_scnn_lr0.12_8x4_160k_cityscapes.py demo/end2end.engine --backend tensorrt --eval cityscapes
How to set `InferenceEngine_DIR` if the openvino backend was installed with `pip`
Could we add it into the docs?
20:15:27 Step 16/17 : RUN mkdir build && cd build && cmake .. -DMMDEPLOY_BUILD_SDK=ON -DCMAKE_CXX_COMPILER=g++-7 -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} -DMMDEPLOY_TARGET_DEVICES="cpu" -DMMDEPLOY_TARGET_BACKENDS=${BACKEND} -DMMDEPLOY_CODEBASES=all -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON && cmake --build . -- -j4 && cmake --install .
20:15:27 ---> Running in 305ad1e0ff77
20:15:28 -- CMAKE_INSTALL_PREFIX: /opt/mmdeploy/build/install
20:15:28 -- The C compiler identification is GNU 7.5.0
20:15:28 -- The CXX compiler identification is GNU 7.5.0
20:15:28 -- Detecting C compiler ABI info
20:15:28 -- Detecting C compiler ABI info - done
20:15:28 -- Check for working C compiler: /usr/bin/cc - skipped
20:15:28 -- Detecting C compile features
20:15:28 -- Detecting C compile features - done
20:15:28 -- Detecting CXX compiler ABI info
20:15:28 -- Detecting CXX compiler ABI info - done
20:15:28 -- Check for working CXX compiler: /usr/bin/g++-7 - skipped
20:15:28 -- Detecting CXX compile features
20:15:28 -- Detecting CXX compile features - done
20:15:28 -- spdlog is found
20:15:28 -- Found OpenCV: /usr (found version "3.2.0")
20:15:28 CMake Error at csrc/net/openvino/CMakeLists.txt:7 (find_package):
20:15:28 By not providing "FindInferenceEngine.cmake" in CMAKE_MODULE_PATH this
20:15:28 project has asked CMake to find a package configuration file provided by
20:15:28 "InferenceEngine", but CMake did not find one.
20:15:28
20:15:28 Could not find a package configuration file provided by "InferenceEngine"
20:15:28 with any of the following names:
20:15:28
20:15:28 InferenceEngineConfig.cmake
20:15:28 inferenceengine-config.cmake
20:15:28
20:15:28 Add the installation prefix of "InferenceEngine" to CMAKE_PREFIX_PATH or
20:15:28 set "InferenceEngine_DIR" to a directory containing one of the above files.
20:15:28 If "InferenceEngine" provides a separate development package or SDK, be
20:15:28 sure it has been installed.
[Enhancement] Load OpenVINO model from stream.
Hi, @SemyonBevzuk
We are working on integrating OpenVINO into our SDK. It seems that we can not load CNNNetwork from a stream. And export ExecutableNetwork is not implemented. So we have to create a temp folder and load the model from it (which violates our design).
Is there any way to load the model from a stream? Or does OpenVINO have any plan to add the interface?
Ut run failed: `test_can_not_run_onnx2openvino_without_mo`
Thanks for your bug report. We appreciate it a lot.
Checklist
With onnxruntime as the backend.
__________________ test_can_not_run_onnx2openvino_without_mo ___________________
@backend_checker(Backend.OPENVINO)
def test_can_not_run_onnx2openvino_without_mo():
current_environ = dict(os.environ)
os.environ.clear()
is_error = False
try:
from mmdeploy.apis.openvino import onnx2openvino
> onnx2openvino({}, ['output'], 'tmp.onnx', '/tmp')
tests/test_apis/test_onnx2openvino.py:99:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
mmdeploy/backend/openvino/onnx2openvino.py:89: in onnx2openvino
mo_output = run(command, capture_output=True, shell=True, check=True)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
input = None, capture_output = True, timeout = None, check = True
popenargs = ('mo --input_model="tmp.onnx" --output_dir="/tmp" --output="output" --input="" --input_shape="" --disable_fusing ',)
kwargs = {'shell': True, 'stderr': -1, 'stdout': -1}
process = <subprocess.Popen object at 0x7f5735041cd0>, stdout = b''
stderr = b'[ ERROR ] The "tmp.onnx" is not existing file\n', retcode = 1
def run(*popenargs,
input=None, capture_output=False, timeout=None, check=False, **kwargs):
"""Run command with arguments and return a CompletedProcess instance.
The returned instance will have attributes args, returncode, stdout and
stderr. By default, stdout and stderr are not captured, and those attributes
will be None. Pass stdout=PIPE and/or stderr=PIPE in order to capture them.
If check is True and the exit code was non-zero, it raises a
CalledProcessError. The CalledProcessError object will have the return code
in the returncode attribute, and output & stderr attributes if those streams
were captured.
If timeout is given, and the process takes too long, a TimeoutExpired
exception will be raised.
There is an optional argument "input", allowing you to
pass bytes or a string to the subprocess's stdin. If you use this argument
you may not also use the Popen constructor's "stdin" argument, as
it will be used internally.
By default, all communication is in bytes, and therefore any "input" should
be bytes, and the stdout and stderr will be bytes. If in text mode, any
"input" should be a string, and stdout and stderr will be strings decoded
according to locale encoding, or by "encoding" if set. Text mode is
triggered by setting any of text, encoding, errors or universal_newlines.
The other arguments are the same as for the Popen constructor.
"""
if input is not None:
if kwargs.get('stdin') is not None:
raise ValueError('stdin and input arguments may not both be used.')
kwargs['stdin'] = PIPE
if capture_output:
if kwargs.get('stdout') is not None or kwargs.get('stderr') is not None:
raise ValueError('stdout and stderr arguments may not be used '
'with capture_output.')
kwargs['stdout'] = PIPE
kwargs['stderr'] = PIPE
with Popen(*popenargs, **kwargs) as process:
try:
stdout, stderr = process.communicate(input, timeout=timeout)
except TimeoutExpired as exc:
process.kill()
if _mswindows:
# Windows accumulates the output in a single blocking
# read() call run on child threads, with the timeout
# being done in a join() on those threads. communicate()
# _after_ kill() is required to collect that and add it
# to the exception.
exc.stdout, exc.stderr = process.communicate()
else:
# POSIX _communicate already populated the output so
# far into the TimeoutExpired exception.
process.wait()
raise
except: # Including KeyboardInterrupt, communicate handled that.
process.kill()
# We don't call process.wait() as .__exit__ does that for us.
raise
retcode = process.poll()
if check and retcode:
> raise CalledProcessError(retcode, process.args,
output=stdout, stderr=stderr)
E subprocess.CalledProcessError: Command 'mo --input_model="tmp.onnx" --output_dir="/tmp" --output="output" --input="" --input_shape="" --disable_fusing ' returned non-zero exit status 1.
/usr/lib/python3.8/subprocess.py:516: CalledProcessError
Describe the bug
A clear and concise description of what the bug is.
Reproduction
- What command or script did you run?
A placeholder for the command.
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
ubuntu_1804_py_38_torch_190_ort_master:20220121
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
[doc] Install mmdeploy
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
或者 pip install -e . [optional]。其中，[optional]可以填写all, tests, build, optional
in https://mmdeploy.readthedocs.io/zh_CN/latest/build.html#id3 or https://mmdeploy.readthedocs.io/en/latest/build.html
The installation following pip install -e . tests or pip install -e . all will be not work
Reproduction
- What command or script did you run?
A placeholder for the command.
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
- Please run
python tools/check_env.pyto collect necessary environment information and paste it here. - You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as
$PATH,$LD_LIBRARY_PATH,$PYTHONPATH, etc.)
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
Issue building MMDeploy with latest ppl.cv master
Hello
Thank you for this project. I have been attempting to build MMDeploy according to the documentation.
Specifically, I am trying to build the SDK with TensorRT support, according to the documentation listed here:
https://mmdeploy.readthedocs.io/en/latest/build.html
I have built ppl.cv using ./build.sh cuda, as instructed (latest version from master). All required dependencies are also installed.
I run the build as outlined here (adapted from the documentation). I rely on cmake to detect Tensorrt and this is working correctly.
cd MMDeploy
mkdir build && cd build
cmake .. \
-DMMDEPLOY_BUILD_SDK=ON \
-Dpplcv_DIR=/home/user/project/ppl.cv/cuda-build/install/lib/cmake/ppl \
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
-DMMDEPLOY_TARGET_BACKENDS=trt \
-DMMDEPLOY_CODEBASES=all
cmake --build . -- -j$(nproc) && cmake --install .
When building MMDeploy, however, I get the following error:
[ 62%] Building CUDA object csrc/preprocess/cuda/CMakeFiles/mmdeploy_cuda_transform_impl.dir/crop.cu.o
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp: In member function ‘mmdeploy::Result<void> mmdeploy::cuda::ResizeImpl::ResizeLinear(const mmdeploy::Tensor&, mmdeploy::Tensor&, cudaStream_t)’:
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:51:30: error: ‘ResizeLinear’ is not a member of ‘ppl::cv::cuda’
51 | ret = ppl::cv::cuda::ResizeLinear<uint8_t, 1>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:51:50: error: expected primary-expression before ‘,’ token
51 | ret = ppl::cv::cuda::ResizeLinear<uint8_t, 1>(stream, h, w, w * c, input, dst_h, dst_w,
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:52:72: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
52 | dst_w * c, output);
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:54:30: error: ‘ResizeLinear’ is not a member of ‘ppl::cv::cuda’
54 | ret = ppl::cv::cuda::ResizeLinear<uint8_t, 3>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:54:50: error: expected primary-expression before ‘,’ token
54 | ret = ppl::cv::cuda::ResizeLinear<uint8_t, 3>(stream, h, w, w * c, input, dst_h, dst_w,
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:55:72: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
55 | dst_w * c, output);
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:57:30: error: ‘ResizeLinear’ is not a member of ‘ppl::cv::cuda’
57 | ret = ppl::cv::cuda::ResizeLinear<uint8_t, 4>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:57:50: error: expected primary-expression before ‘,’ token
57 | ret = ppl::cv::cuda::ResizeLinear<uint8_t, 4>(stream, h, w, w * c, input, dst_h, dst_w,
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:58:72: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
58 | dst_w * c, output);
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:67:30: error: ‘ResizeLinear’ is not a member of ‘ppl::cv::cuda’
67 | ret = ppl::cv::cuda::ResizeLinear<float, 1>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:67:43: error: expected primary-expression before ‘float’
67 | ret = ppl::cv::cuda::ResizeLinear<float, 1>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:70:30: error: ‘ResizeLinear’ is not a member of ‘ppl::cv::cuda’
70 | ret = ppl::cv::cuda::ResizeLinear<float, 3>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:70:43: error: expected primary-expression before ‘float’
70 | ret = ppl::cv::cuda::ResizeLinear<float, 3>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:73:30: error: ‘ResizeLinear’ is not a member of ‘ppl::cv::cuda’
73 | ret = ppl::cv::cuda::ResizeLinear<float, 4>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:73:43: error: expected primary-expression before ‘float’
73 | ret = ppl::cv::cuda::ResizeLinear<float, 4>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp: In member function ‘mmdeploy::Result<void> mmdeploy::cuda::ResizeImpl::ResizeNearest(const mmdeploy::Tensor&, mmdeploy::Tensor&, cudaStream_t)’:
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:99:30: error: ‘ResizeNearestPoint’ is not a member of ‘ppl::cv::cuda’
99 | ret = ppl::cv::cuda::ResizeNearestPoint<uint8_t, 1>(stream, h, w, w * c, input, dst_h,
| ^~~~~~~~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:99:56: error: expected primary-expression before ‘,’ token
99 | ret = ppl::cv::cuda::ResizeNearestPoint<uint8_t, 1>(stream, h, w, w * c, input, dst_h,
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:100:85: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
100 | dst_w, dst_w * c, output);
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:102:30: error: ‘ResizeNearestPoint’ is not a member of ‘ppl::cv::cuda’
102 | ret = ppl::cv::cuda::ResizeNearestPoint<uint8_t, 3>(stream, h, w, w * c, input, dst_h,
| ^~~~~~~~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:102:56: error: expected primary-expression before ‘,’ token
102 | ret = ppl::cv::cuda::ResizeNearestPoint<uint8_t, 3>(stream, h, w, w * c, input, dst_h,
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:103:85: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
103 | dst_w, dst_w * c, output);
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:105:30: error: ‘ResizeNearestPoint’ is not a member of ‘ppl::cv::cuda’
105 | ret = ppl::cv::cuda::ResizeNearestPoint<uint8_t, 4>(stream, h, w, w * c, input, dst_h,
| ^~~~~~~~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:105:56: error: expected primary-expression before ‘,’ token
105 | ret = ppl::cv::cuda::ResizeNearestPoint<uint8_t, 4>(stream, h, w, w * c, input, dst_h,
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:106:85: error: ISO C++ forbids comparison between pointer and integer [-fpermissive]
106 | dst_w, dst_w * c, output);
| ^
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:115:30: error: ‘ResizeNearestPoint’ is not a member of ‘ppl::cv::cuda’
115 | ret = ppl::cv::cuda::ResizeNearestPoint<float, 1>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:115:49: error: expected primary-expression before ‘float’
115 | ret = ppl::cv::cuda::ResizeNearestPoint<float, 1>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:118:30: error: ‘ResizeNearestPoint’ is not a member of ‘ppl::cv::cuda’
118 | ret = ppl::cv::cuda::ResizeNearestPoint<float, 3>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:118:49: error: expected primary-expression before ‘float’
118 | ret = ppl::cv::cuda::ResizeNearestPoint<float, 3>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:121:30: error: ‘ResizeNearestPoint’ is not a member of ‘ppl::cv::cuda’
121 | ret = ppl::cv::cuda::ResizeNearestPoint<float, 4>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~~~~~~~~~~~~~~
/home/user/project/MMDeploy/csrc/preprocess/cuda/resize_impl.cpp:121:49: error: expected primary-expression before ‘float’
121 | ret = ppl::cv::cuda::ResizeNearestPoint<float, 4>(stream, h, w, w * c, input, dst_h, dst_w,
| ^~~~~
It looks like ResizeLinear() and ResizeNearestPoint() has been replaced with Resize(.., ppl::cv::INTERPOLATION_TYPE_xx) where an optional input argument has been added.
The solution should be to replace the above functions with Resize(..) with the last argument set according to the resize type.
I will try to make a PR, if the above solution works as expected.
Build ONNXRUNTIME ops failed
Thanks for your bug report. We appreciate it a lot.
Describe the bug
A clear and concise description of what the bug is.
Reproduction
- What command or script did you run?
2022-01-19T05:37:30.415Z] Step 16/17 : RUN mkdir build && cd build && cmake .. -DMMDEPLOY_BUILD_SDK=ON -DCMAKE_CXX_COMPILER=g++-7 -DONNXRUNTIME_DIR=${ONNXRUNTIME_DIR} -DMMDEPLOY_TARGET_DEVICES="cpu" -DMMDEPLOY_TARGET_BACKENDS=${BACKEND} -DMMDEPLOY_CODEBASES=all -DMMDEPLOY_BUILD_SDK_PYTHON_API=ON && cmake --build . -- -j4 && cmake --install .
[2022-01-19T05:37:30.415Z] ---> Running in 04f4ea5c6c39
[2022-01-19T05:37:30.415Z] -- CMAKE_INSTALL_PREFIX: /opt/mmdeploy/build/install
[2022-01-19T05:37:30.415Z] -- The C compiler identification is GNU 7.5.0
[2022-01-19T05:37:30.415Z] -- The CXX compiler identification is GNU 7.5.0
[2022-01-19T05:37:30.415Z] -- Detecting C compiler ABI info
[2022-01-19T05:37:30.415Z] -- Detecting C compiler ABI info - done
[2022-01-19T05:37:30.415Z] -- Check for working C compiler: /usr/bin/cc - skipped
[2022-01-19T05:37:30.415Z] -- Detecting C compile features
[2022-01-19T05:37:30.415Z] -- Detecting C compile features - done
[2022-01-19T05:37:30.415Z] -- Detecting CXX compiler ABI info
[2022-01-19T05:37:30.415Z] -- Detecting CXX compiler ABI info - done
[2022-01-19T05:37:30.415Z] -- Check for working CXX compiler: /usr/bin/g++-7 - skipped
[2022-01-19T05:37:30.415Z] -- Detecting CXX compile features
[2022-01-19T05:37:30.415Z] -- Detecting CXX compile features - done
[2022-01-19T05:37:30.415Z] -- Build ONNXRUNTIME custom ops.
[2022-01-19T05:37:30.415Z] CMake Error at cmake/opencv.cmake:2 (find_package):
[2022-01-19T05:37:30.415Z] By not providing "FindOpenCV.cmake" in CMAKE_MODULE_PATH this project has
[2022-01-19T05:37:30.415Z] asked CMake to find a package configuration file provided by "OpenCV", but
[2022-01-19T05:37:30.415Z] CMake did not find one.
[2022-01-19T05:37:30.415Z]
[2022-01-19T05:37:30.415Z] Could not find a package configuration file provided by "OpenCV" with any
[2022-01-19T05:37:30.415Z] of the following names:
[2022-01-19T05:37:30.415Z]
[2022-01-19T05:37:30.415Z] OpenCVConfig.cmake
[2022-01-19T05:37:30.415Z] opencv-config.cmake
[2022-01-19T05:37:30.415Z]
[2022-01-19T05:37:30.415Z] Add the installation prefix of "OpenCV" to CMAKE_PREFIX_PATH or set
[2022-01-19T05:37:30.415Z] "OpenCV_DIR" to a directory containing one of the above files. If "OpenCV"
[2022-01-19T05:37:30.415Z] provides a separate development package or SDK, be sure it has been
[2022-01-19T05:37:30.415Z] installed.
[2022-01-19T05:37:30.415Z] Call Stack (most recent call first):
[2022-01-19T05:37:30.415Z] csrc/preprocess/cpu/CMakeLists.txt:5 (include)
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
- Please run
python tools/check_env.pyto collect necessary environment information and paste it here. - You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as
$PATH,$LD_LIBRARY_PATH,$PYTHONPATH, etc.)
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
RuntimeError: dictionary keys changed during iteration
raised error when convert mmdetection model to openvino
command
python tools/deploy.py configs/mmdet/detection/detection_openvino_dynamic-800x1344.py mmdetection/configs/retinanet/retinanet_r50_fpn_1x_coco.py mmdetection/checkpoints/retinanet/retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth det.jpg --work-dir ../deploy_result --device cpu
env
- TorchVision: 0.11.2+cu102
- OpenCV: 4.5.5
- MMCV: 1.4.2
- MMCV Compiler: GCC 8.3
- MMCV CUDA Compiler: not available
- MMDeployment: 0.1.0+fbce75e
- onnxruntime: None ops_is_avaliable : False
- tensorrt: None ops_is_avaliable : False
- ncnn: None ops_is_avaliable : False
- pplnn_is_avaliable: False
- openvino_is_avaliable: True
Error traceback
2022-01-12 02:16:04,908 - mmdeploy - INFO - torch2onnx start.
load checkpoint from local path: /tmp/mmlab_deploy/mmdetection/checkpoints/retinanet/retinanet_r50_fpn_1x_coco_20200130-c2398f9e.pth
/usr/local/lib/python3.8/site-packages/mmdet/datasets/utils.py:65: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
warnings.warn(
/home/user/.local/lib/python3.8/site-packages/torch/onnx/utils.py:97: UserWarning: `strip_doc_string' is deprecated and ignored. Will be removed in next PyTorch release. It's combined with `verbose' argument now.
warnings.warn("`strip_doc_string' is deprecated and ignored. Will be removed in "
/home/user/MMDeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/models/dense_heads/base_dense_head.py:94: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/home/user/MMDeploy/mmdeploy/pytorch/functions/topk.py:27: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
k = torch.tensor(k, device=input.device, dtype=torch.long)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(0) == bboxes.size(0)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(1) == bboxes.size(1)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:97: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
deltas.size(-1) // 4)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:98: UserWarning: __floordiv__ is deprecated, and its behavior will change in a future version of pytorch. It currently rounds toward 0 (like the 'trunc' function NOT 'floor'). This results in incorrect rounding for negative values. To keep the current behavior, use torch.div(a, b, rounding_mode='trunc'), or for actual floor division, use torch.div(a, b, rounding_mode='floor').
stds = deltas.new_tensor(stds).view(1, -1).repeat(1, deltas.size(-1) // 4)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/deploy/utils.py:45: TracerWarning: Using len to get tensor shape might cause the trace to be incorrect. Recommended usage would be tensor.shape[0]. Passing a tensor of different shape might lead to errors or silently give incorrect results.
assert len(max_shape) == 2, '`max_shape` should be [h, w]'
/home/user/MMDeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:109: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
/home/user/MMDeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:110: TracerWarning: torch.tensor results are registered as constants in the trace. You can safely ignore this warning if you use this function to create tensors out of constant variables that would be the same every time you call this function. In any other case, this might cause the trace to be incorrect.
score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
/home/user/MMDeploy/mmdeploy/mmcv/ops/nms.py:40: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
score_threshold = float(score_threshold)
/home/user/MMDeploy/mmdeploy/mmcv/ops/nms.py:41: TracerWarning: Converting a tensor to a Python float might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
iou_threshold = float(iou_threshold)
/home/user/.local/lib/python3.8/site-packages/mmcv/ops/nms.py:159: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(1) == 4
/home/user/.local/lib/python3.8/site-packages/mmcv/ops/nms.py:160: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert boxes.size(0) == scores.size(0)
/home/user/.local/lib/python3.8/site-packages/mmcv/ops/nms.py:29: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if max_num > 0:
/home/user/.local/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:2815: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
warnings.warn("Exporting aten::index operator of advanced indexing in opset " +
2022-01-12 02:16:33,191 - mmdeploy - INFO - torch2onnx success.
2022-01-12 02:16:33,311 - mmdeploy - INFO - onnx2openvino with ../deploy_result/end2end.onnx start.
2022-01-12 02:16:36,469 - mmdeploy - INFO - Args for Model Optimizer: mo --input_model="../deploy_result/end2end.onnx" --output_dir="../deploy_result" --output="dets,labels" --input="input" --input_shape="[1, 3, 800, 1344]" --disable_fusing
2022-01-12 02:18:01,536 - mmdeploy - INFO - Model Optimizer arguments:
Common parameters:
- Path to the Input Model: /home/user/MMDeploy/../deploy_result/end2end.onnx
- Path for generated IR: /home/user/MMDeploy/../deploy_result
- IR output name: end2end
- Log level: ERROR
- Batch: Not specified, inherited from the model
- Input layers: input
- Output layers: dets,labels
- Input shapes: [1, 3, 800, 1344]
- Mean values: Not specified
- Scale values: Not specified
- Scale factor: Not specified
- Precision of IR: FP32
- Enable fusing: False
- Enable grouped convolutions fusing: True
- Move mean values to preprocess section: None
- Reverse input channels: False
ONNX specific parameters:
- Inference Engine found in: /home/user/.local/lib/python3.8/site-packages/openvino
Inference Engine version: 2021.4.2-3976-0943ed67223-refs/pull/539/head
Model Optimizer version: 2021.4.2-3976-0943ed67223-refs/pull/539/head
[ SUCCESS ] Generated IR version 10 model.
[ SUCCESS ] XML file: /home/user/deploy_result/end2end.xml
[ SUCCESS ] BIN file: /home/user/deploy_result/end2end.bin
[ SUCCESS ] Total execution time: 84.31 seconds.
[ SUCCESS ] Memory consumed: 674 MB.
2022-01-12 02:18:01,537 - mmdeploy - INFO - Successfully exported OpenVINO model: ../deploy_result/end2end.xml
2022-01-12 02:18:01,803 - mmdeploy - INFO - onnx2openvino with ../deploy_result/end2end.onnx success.
2022-01-12 02:18:01,803 - mmdeploy - INFO - visualize openvino model start.
/usr/local/lib/python3.8/site-packages/mmdet/datasets/utils.py:65: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
warnings.warn(
dict_keys(['TopK_1171.0', 'TopK_1305.0', 'TopK_564.0', 'TopK_656.0', 'TopK_748.0', 'TopK_840.0', 'TopK_932.0', 'dets', 'labels'])
2022-01-12:02:18:18,root ERROR [utils.py:41] dictionary keys changed during iteration
Traceback (most recent call last):
File "/home/user/MMDeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper
result = target(*args, **kwargs)
File "/home/user/MMDeploy/mmdeploy/apis/visualize.py", line 60, in visualize_model
result = task_processor.run_inference(model, model_inputs)[0]
File "/home/user/MMDeploy/mmdeploy/codebase/mmdet/deploy/object_detection.py", line 188, in run_inference
return model(**model_inputs, return_loss=False, rescale=True)
File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/user/MMDeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 186, in forward
outputs = self.forward_test(input_img, img_metas, *args, **kwargs)
File "/home/user/MMDeploy/mmdeploy/codebase/mmdet/deploy/object_detection_model.py", line 261, in forward_test
outputs = self.wrapper({self.input_name: imgs})
File "/home/user/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
return forward_call(*input, **kwargs)
File "/home/user/MMDeploy/mmdeploy/backend/openvino/wrapper.py", line 130, in forward
outputs = self.__process_outputs(outputs)
File "/home/user/MMDeploy/mmdeploy/backend/openvino/wrapper.py", line 111, in __process_outputs
for output_name in outputs.keys():
RuntimeError: dictionary keys changed during iteration
2022-01-12 02:18:19,123 - mmdeploy - ERROR - visualize openvino model failed.
Redefine SDK's logger macro
Describe the feature
Redefine SDK's logger macro by adding a prefix
Motivation
SDK defines the DEBUG, INFO, WARN, ERROR, CRITICAL log level.
Those keywords are so common that they probably bring conflicts when introducing 3rdparty libraries.
Therefore, it's better to rename them by adding a prefix, like 'MMDEPLOY_'
TensorRT ".engine" python inference
Hi,
Is there any documentation about how to deploy TensorRT engine file for python inference??
Thanks!
problem for install
(openmmlab) root@Ailab:~/nlp/mmdeploy/build# make -j$(nproc)
Consolidate compiler generated dependencies of target mmdeploy_onnxruntime_ops_obj
[ 16%] Building CXX object csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/grid_sample/grid_sample.cpp.o
[ 33%] Building CXX object csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/roi_align/roi_align.cpp.o
[ 50%] Building CXX object csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/modulated_deform_conv/modulated_deform_conv.cpp.o
In file included from /root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/grid_sample/grid_sample.cpp:9:0:
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:34:25: error: the value of ‘mmdeploy::__domain_mmdeployGridSampleOp’ is not usable in a constant expression
static OrtOpsRegistry<domain##domain##name, name> ort_ops_registry##domain##name {}
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/grid_sample/grid_sample.cpp:293:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmdeploy, GridSampleOp);
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:33:15: note: ‘mmdeploy::__domain_mmdeployGridSampleOp’ was not declared ‘constexpr’
static char _domain##domain##name[] = #domain;
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/grid_sample/grid_sample.cpp:293:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmdeploy, GridSampleOp);
^
In file included from /root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/modulated_deform_conv/modulated_deform_conv.cpp:7:0:
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:34:25: error: the value of ‘mmdeploy::__domain_mmdeployMMCVModulatedDeformConvOp’ is not usable in a constant expression
static OrtOpsRegistry<domain##domain##name, name> ort_ops_registry##domain##name {}
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/modulated_deform_conv/modulated_deform_conv.cpp:269:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmdeploy, MMCVModulatedDeformConvOp);
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:33:15: note: ‘mmdeploy::__domain_mmdeployMMCVModulatedDeformConvOp’ was not declared ‘constexpr’
static char _domain##domain##name[] = #domain;
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/modulated_deform_conv/modulated_deform_conv.cpp:269:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmdeploy, MMCVModulatedDeformConvOp);
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:34:25: error: the value of ‘mmdeploy::__domain_mmcvMMCVModulatedDeformConvOp’ is not usable in a constant expression
static OrtOpsRegistry<domain##domain##name, name> ort_ops_registry##domain##name {}
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/modulated_deform_conv/modulated_deform_conv.cpp:270:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmcv, MMCVModulatedDeformConvOp);
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:33:15: note: ‘mmdeploy::__domain_mmcvMMCVModulatedDeformConvOp’ was not declared ‘constexpr’
static char _domain##domain##name[] = #domain;
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/modulated_deform_conv/modulated_deform_conv.cpp:270:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmcv, MMCVModulatedDeformConvOp);
^
In file included from /root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/roi_align/roi_align.cpp:6:0:
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:34:25: error: the value of ‘mmdeploy::__domain_mmdeployMMCVRoiAlignCustomOp’ is not usable in a constant expression
static OrtOpsRegistry<domain##domain##name, name> ort_ops_registry##domain##name {}
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/roi_align/roi_align.cpp:254:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmdeploy, MMCVRoiAlignCustomOp);
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/common/ort_utils.h:33:15: note: ‘mmdeploy::__domain_mmdeployMMCVRoiAlignCustomOp’ was not declared ‘constexpr’
static char _domain##domain##name[] = #domain;
^
/root/nlp/mmdeploy/csrc/backend_ops/onnxruntime/roi_align/roi_align.cpp:254:1: note: in expansion of macro ‘REGISTER_ONNXRUNTIME_OPS’
REGISTER_ONNXRUNTIME_OPS(mmdeploy, MMCVRoiAlignCustomOp);
^
csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/build.make:89: recipe for target 'csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/grid_sample/grid_sample.cpp.o' failed
make[2]: *** [csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/grid_sample/grid_sample.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/build.make:103: recipe for target 'csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/modulated_deform_conv/modulated_deform_conv.cpp.o' failed
make[2]: *** [csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/modulated_deform_conv/modulated_deform_conv.cpp.o] Error 1
csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/build.make:131: recipe for target 'csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/roi_align/roi_align.cpp.o' failed
make[2]: *** [csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/roi_align/roi_align.cpp.o] Error 1
CMakeFiles/Makefile2:129: recipe for target 'csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/all' failed
make[1]: *** [csrc/backend_ops/onnxruntime/CMakeFiles/mmdeploy_onnxruntime_ops_obj.dir/all] Error 2
Makefile:135: recipe for target 'all' failed
make: *** [all] Error 2
INVALID_GRAPH: Assertion failed: ctx->tensors().count(inputName)
Running the example gives the error as titled.
python ./tools/deploy.py \ configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \ $PATH_TO_MMDET/configs/yolo/yolov3_d53_mstrain-608_273e_coco.py \ $PATH_TO_MMDET/checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.pth \ $PATH_TO_MMDET/demo/demo.jpg \ --work-dir work_dir \ --show \ --device cuda:0
My environment:
- PyTorch 1.8.0
- TensorRT 7.1.3
It seems the TensorRT version is lower than required. Does MMDepoly only support TensorRT 8?
ImportError: cannot import name '_get_tensor_dim_size'
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
A clear and concise description of what the bug is.
Reproduction
- What command or script did you run?
[2022-01-19T07:54:57.643Z] Removing intermediate container 1dac7717b7f9
[2022-01-19T07:54:57.643Z] ---> 1153d2075ef3
[2022-01-19T07:54:57.643Z] Step 17/17 : RUN python3 tools/check_env.py
[2022-01-19T07:54:57.911Z] ---> Running in ed13e71986f7
[2022-01-19T07:55:01.230Z] Traceback (most recent call last):
[2022-01-19T07:55:01.231Z] File "tools/check_env.py", line 5, in <module>
[2022-01-19T07:55:01.231Z] import mmdeploy
[2022-01-19T07:55:01.231Z] File "/opt/mmdeploy/mmdeploy/__init__.py", line 7, in <module>
[2022-01-19T07:55:01.231Z] importlib.import_module('mmdeploy.pytorch')
[2022-01-19T07:55:01.231Z] File "/usr/lib/python3.6/importlib/__init__.py", line 126, in import_module
[2022-01-19T07:55:01.231Z] return _bootstrap._gcd_import(name[level:], package, level)
[2022-01-19T07:55:01.231Z] File "/opt/mmdeploy/mmdeploy/pytorch/__init__.py", line 3, in <module>
[2022-01-19T07:55:01.231Z] from .ops import * # noqa: F401,F403
[2022-01-19T07:55:01.231Z] File "/opt/mmdeploy/mmdeploy/pytorch/ops/__init__.py", line 6, in <module>
[2022-01-19T07:55:01.231Z] from .instance_norm import instance_norm__tensorrt
[2022-01-19T07:55:01.231Z] File "/opt/mmdeploy/mmdeploy/pytorch/ops/instance_norm.py", line 6, in <module>
[2022-01-19T07:55:01.231Z] from torch.onnx.symbolic_helper import (_get_tensor_dim_size, _get_tensor_rank,
[2022-01-19T07:55:01.231Z] ImportError: cannot import name '_get_tensor_dim_size'
[2022-01-19T07:55:01.808Z] The command '/bin/sh -c python3 tools/check_env.py' returned a non-zero code: 1
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
docker build -t mmdeploy_ubuntu_1804_py_37_torch_160_ort_master:20220119 --build-arg IMAGE=ubuntu_1804_py_37_torch_160 --build-arg BACKEND=ort --build-arg TAG=v1 --
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
How to deploy with --device cuda:1 ?
It works when I use cuda:0 while it not works when I use cuda:1. Same on inferecing.
How can I use both cuda with TensorRT?
convertion:
python
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/tools/deploy.py
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/configs/mmdet/instance-seg/instance-seg_tensorrt_dynamic-320x320-1344x1344.py
/home/aiuser/workspace/mmdetection/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py
/home/aiuser/workspace/mmdetection/checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth
/home/aiuser/workspace/mmdetection/demo/demo.jpg
--work-dir /home/aiuser/workspace/lab_mmdeploy/work_dir
--device cuda:1
--dump-info \
log:
[01/26/2022-17:19:04] [TRT] [I] Successfully created plugin: MMCVMultiLevelRoiAlign
[01/26/2022-17:19:04] [TRT] [W] Output type must be INT32 for shape outputs
[01/26/2022-17:19:04] [TRT] [W] Output type must be INT32 for shape outputs
[01/26/2022-17:19:06] [TRT] [E] 9: [backendBuilderConfig.cpp::validateBasicAssumptions::34] Error Code 9: Internal Error (Builder was created on device different than current device.)
2022-01-26:17:19:06,root ERROR [utils.py:41] Failed to create TensorRT engine
Traceback (most recent call last):
File "/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/utils/utils.py", line 36, in target_wrapper
result = target(*args, **kwargs)
File "/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 72, in onnx2tensorrt
device_id=device_id)
File "/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/backend/tensorrt/utils.py", line 116, in create_trt_engine
assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-01-26 17:19:06,272 - mmdeploy - ERROR - onnx2tensorrt of /home/aiuser/workspace/lab_mmdeploy/work_dir_cuda1/end2end.onnx failed.
(mmdeploy) aiuser@aiuser-System-Product-Name:~/workspace/lab_mmdeploy$
Could not find any implementation for node
env:
cuda 11.1
torch 1.8.2
python 3.8.12
tensorrt 8.2.2.1
====== log start ======
(mmdeploy_torch182) aiuser@aiuser-System-Product-Name:/workspace/lab_mmdeploy/work_dir$ python /home/aiuser/workspace/lab_mmdeploy/MMDeploy/tools/deploy.py /home/aiuser/workspace/lab_mmdeploy/MMDeploy/configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py /home/aiuser/workspace/mmdetection/configs/mask_rcnn/mask_rcnn_r50_fpn_1x_coco.py /home/aiuser/workspace/mmdetection/checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth /home/aiuser/workspace/mmdetection/demo/COCO_test2014_000000118888.jpg --work-dir /home/aiuser/workspace/lab_mmdeploy/work_dir --device cuda:0 --dump-info
WARNING: CPU random generator seem to be failing, disabling hardware random number generation
WARNING: RDRND generated: 0xffffffff 0xffffffff 0xffffffff 0xffffffff
/home/aiuser/miniconda3/envs/mmdeploy_torch182/lib/python3.8/site-packages/mmdet/datasets/utils.py:65: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
warnings.warn(
2022-01-06:14:51:37,root INFO [deploy.py:72] torch2onnx start.
WARNING: CPU random generator seem to be failing, disabling hardware random number generation
WARNING: RDRND generated: 0xffffffff 0xffffffff 0xffffffff 0xffffffff
load checkpoint from local path: /home/aiuser/workspace/mmdetection/checkpoints/mask_rcnn_r50_fpn_1x_coco_20200205-d4b0c5d6.pth
/home/aiuser/miniconda3/envs/mmdeploy_torch182/lib/python3.8/site-packages/mmdet/datasets/utils.py:65: UserWarning: "ImageToTensor" pipeline is replaced by "DefaultFormatBundle" for batch inference. It is recommended to manually replace it in the test data pipeline in your config file.
warnings.warn(
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/core/optimizers/function_marker.py:158: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
ys_shape = tuple(int(s) for s in ys.shape)
/home/aiuser/miniconda3/envs/mmdeploy_torch182/lib/python3.8/site-packages/mmdet/models/dense_heads/anchor_head.py:123: UserWarning: DeprecationWarning: anchor_generator is deprecated, please use "prior_generator" instead
warnings.warn('DeprecationWarning: anchor_generator is deprecated, '
/home/aiuser/miniconda3/envs/mmdeploy_torch182/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:333: UserWarning: grid_anchors would be deprecated soon. Please use grid_priors
warnings.warn('grid_anchors would be deprecated soon. '
/home/aiuser/miniconda3/envs/mmdeploy_torch182/lib/python3.8/site-packages/mmdet/core/anchor/anchor_generator.py:369: UserWarning: single_level_grid_anchors would be deprecated soon. Please use single_level_grid_priors
warnings.warn(
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/codebase/mmdet/models/dense_heads/rpn_head.py:77: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/pytorch/functions/topk.py:54: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
if k > size:
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:39: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(0) == bboxes.size(0)
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/codebase/mmdet/core/bbox/delta_xywh_bbox_coder.py:41: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
assert pred_bboxes.size(1) == bboxes.size(1)
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/codebase/mmdet/core/post_processing/bbox_nms.py:167: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
dets, labels = TRTBatchedNMSop.apply(boxes, scores, int(scores.shape[-1]),
/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/mmcv/ops/nms.py:177: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
out_boxes = min(num_boxes, after_topk)
/home/aiuser/miniconda3/envs/mmdeploy_torch182/lib/python3.8/site-packages/torch/onnx/symbolic_opset9.py:2620: UserWarning: Exporting aten::index operator of advanced indexing in opset 11 is achieved by combination of multiple ONNX operators, including Reshape, Transpose, Concat, and Gather. If indices include negative values, the exported graph will produce incorrect results.
warnings.warn("Exporting aten::index operator of advanced indexing in opset " +
2022-01-06:14:51:59,root INFO [deploy.py:86] torch2onnx success.
2022-01-06:14:52:00,root INFO [deploy.py:72] onnx2tensorrt of /home/aiuser/workspace/lab_mmdeploy/work_dir/end2end.onnx start.
WARNING: CPU random generator seem to be failing, disabling hardware random number generation
WARNING: RDRND generated: 0xffffffff 0xffffffff 0xffffffff 0xffffffff
2022-01-06:14:52:02,root INFO [init_plugins.py:34] Successfully loaded tensorrt plugins from /home/aiuser/workspace/lab_mmdeploy/MMDeploy/build/lib/libmmdeploy_tensorrt_ops.so
[01/06/2022-14:52:02] [TRT] [I] [MemUsageChange] Init CUDA: CPU +323, GPU +0, now: CPU 390, GPU 235 (MiB)
[01/06/2022-14:52:02] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 390 MiB, GPU 235 MiB
[01/06/2022-14:52:03] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 525 MiB, GPU 269 MiB
[01/06/2022-14:52:03] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[01/06/2022-14:52:03] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:05] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:05] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:05] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:05] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:05] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:06] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:06] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[01/06/2022-14:52:06] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[01/06/2022-14:52:06] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[01/06/2022-14:52:06] [TRT] [I] No importer registered for op: MMCVMultiLevelRoiAlign. Attempting to import as plugin.
[01/06/2022-14:52:06] [TRT] [I] Searching for plugin: MMCVMultiLevelRoiAlign, plugin_version: 1, plugin_namespace:
[01/06/2022-14:52:06] [TRT] [I] Successfully created plugin: MMCVMultiLevelRoiAlign
[01/06/2022-14:52:06] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:06] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:06] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:07] [TRT] [W] onnx2trt_utils.cpp:392: One or more weights outside the range of INT32 was clamped
[01/06/2022-14:52:08] [TRT] [W] Tensor DataType is determined at build time for tensors not marked as input or output.
[01/06/2022-14:52:08] [TRT] [I] No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[01/06/2022-14:52:08] [TRT] [I] Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[01/06/2022-14:52:08] [TRT] [I] Successfully created plugin: TRTBatchedNMS
[01/06/2022-14:52:08] [TRT] [I] No importer registered for op: MMCVMultiLevelRoiAlign. Attempting to import as plugin.
[01/06/2022-14:52:08] [TRT] [I] Searching for plugin: MMCVMultiLevelRoiAlign, plugin_version: 1, plugin_namespace:
[01/06/2022-14:52:08] [TRT] [I] Successfully created plugin: MMCVMultiLevelRoiAlign
[01/06/2022-14:52:09] [TRT] [I] No importer registered for op: grid_sampler. Attempting to import as plugin.
[01/06/2022-14:52:09] [TRT] [I] Searching for plugin: grid_sampler, plugin_version: 1, plugin_namespace:
[01/06/2022-14:52:09] [TRT] [I] Successfully created plugin: grid_sampler
[01/06/2022-14:52:09] [TRT] [W] Output type must be INT32 for shape outputs
[01/06/2022-14:52:09] [TRT] [W] Output type must be INT32 for shape outputs
[01/06/2022-14:52:09] [TRT] [W] Output type must be INT32 for shape outputs
[01/06/2022-14:52:09] [TRT] [W] Output type must be INT32 for shape outputs
[01/06/2022-14:52:15] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.3.0
[01/06/2022-14:52:15] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +93, GPU +164, now: CPU 3413, GPU 1473 (MiB)
[01/06/2022-14:52:15] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +397, GPU +172, now: CPU 3810, GPU 1645 (MiB)
[01/06/2022-14:52:15] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[01/06/2022-14:52:28] [TRT] [I] Some tactics do not have sufficient workspace memory to run. Increasing workspace size may increase performance, please check verbose output.
[01/06/2022-14:52:50] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[01/06/2022-14:52:50] [TRT] [W] (# 2 (SHAPE input))
[01/06/2022-14:52:50] [TRT] [W] (# 3 (SHAPE input))
[01/06/2022-14:53:20] [TRT] [W] Myelin graph with multiple dynamic values may have poor performance if they differ. Dynamic values are:
[01/06/2022-14:53:20] [TRT] [W] (# 3 (SHAPE input))
[01/06/2022-14:53:20] [TRT] [W] (# 2 (SHAPE input))
[01/06/2022-14:53:21] [TRT] [W] Skipping tactic 0 due to insuficient memory on requested size of 1459519488 detected for tactic 0.
[01/06/2022-14:53:21] [TRT] [E] 10: [optimizer.cpp::computeCosts::2011] Error Code 10: Internal Error (Could not find any implementation for node {ForeignNode[2337...Reshape_1696]}.)
2022-01-06:14:53:21,root ERROR [deploy.py:67] Failed to create TensorRT engine
Traceback (most recent call last):
File "/home/aiuser/workspace/lab_mmdeploy/MMDeploy/tools/deploy.py", line 62, in target_wrapper
result = target(*args, **kwargs)
File "/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 64, in onnx2tensorrt
engine = create_trt_engine(
File "/home/aiuser/workspace/lab_mmdeploy/MMDeploy/mmdeploy/backend/tensorrt/utils.py", line 115, in create_trt_engine
assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
2022-01-06:14:53:22,root ERROR [deploy.py:83] onnx2tensorrt of /home/aiuser/workspace/lab_mmdeploy/work_dir/end2end.onnx failed.
(mmdeploy_torch182) aiuser@aiuser-System-Product-Name:/workspace/lab_mmdeploy/work_dir$
====== log end======
ps: the node 2337 is a "Gather" op
No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
Describe the bug
I tried the example provided in how to convert model, meet no importer registered for op problems.
Any help?
Reproduction
python ./tools/deploy.py \
configs/mmdet/detection/detection_tensorrt_dynamic-320x320-1344x1344.py \
$PATH_TO_MMDET/configs/yolo/yolov3_d53_mstrain-608_273e_coco.py \
$PATH_TO_MMDET/checkpoints/yolo/yolov3_d53_mstrain-608_273e_coco.pth \
$PATH_TO_MMDET/demo/demo.jpg \
--work-dir work_dir \
--show \
--device cuda:0Environment
INFO:root:TorchVision: 0.9.0a0
INFO:root:OpenCV: 4.4.0
INFO:root:MMCV: 1.4.2
INFO:root:MMCV Compiler: GCC 9.3
INFO:root:MMCV CUDA Compiler: 11.2
INFO:root:MMDeployment: 0.1.0+f693f7c
INFO:root:onnxruntime: 1.6.0 ops_is_avaliable : True
INFO:root:tensorrt: 7.2.2.3 ops_is_avaliable : True
INFO:root:ncnn: None ops_is_avaliable : False
INFO:root:pplnn_is_avaliable: False
INFO:root:openvino_is_avaliable: FalseError traceback
2021-12-31:07:01:57,root INFO [deploy.py:86] torch2onnx success.
2021-12-31:07:01:57,root INFO [deploy.py:72] onnx2tensorrt of work_dir/end2end.onnx start.
2021-12-31:07:02:03,root WARNING [init_plugins.py:37] Could not load the library of tensorrt plugins. Because the file does not exist:
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:226: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/builtin_op_importers.cpp:3010: TensorRT currently uses half_pixel calculation for the pytorch_half_pixel transformation mode. These are equivalent except for interpolations down to 1D.
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/builtin_op_importers.cpp:3010: TensorRT currently uses half_pixel calculation for the pytorch_half_pixel transformation mode. These are equivalent except for interpolations down to 1D.
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] WARNING: /home/jenkins/workspace/OSS/L0_MergeRequest/oss/parsers/onnx/onnx2trt_utils.cpp:254: One or more weights outside the range of INT32 was clamped
[TensorRT] INFO: No importer registered for op: TRTBatchedNMS. Attempting to import as plugin.
[TensorRT] INFO: Searching for plugin: TRTBatchedNMS, plugin_version: 1, plugin_namespace:
[TensorRT] ERROR: INVALID_ARGUMENT: getPluginCreator could not find plugin TRTBatchedNMS version 1
2021-12-31:07:02:05,root ERROR [deploy.py:67] Failed to parse onnx, In node 889 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
Traceback (most recent call last):
File "/data/ysc_data/mmdeploy/tools/deploy.py", line 62, in target_wrapper
result = target(*args, **kwargs)
File "/data/ysc_data/mmdeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 56, in onnx2tensorrt
engine = create_trt_engine(
File "/data/ysc_data/mmdeploy/mmdeploy/backend/tensorrt/utils.py", line 75, in create_trt_engine
raise RuntimeError(f'Failed to parse onnx, {error_msgs}')
RuntimeError: Failed to parse onnx, In node 889 (importFallbackPluginImporter): UNSUPPORTED_NODE: Assertion failed: creator && "Plugin not found, are the plugin name, version, and namespace correct?"
2021-12-31:07:02:05,root ERROR [deploy.py:83] onnx2tensorrt of work_dir/end2end.onnx failed.problem about convert deeplabv3+ to tensorrt engine
envs version：
import torch
import torchvision
import mmcv
import tensorrt
torch.version
'1.8.1+cu111'
torchvision.version
'0.9.1+cu111'
mmcv.version
'1.4.2'
tensorrt.version
'8.2.1.8'
cuda：2080Ti
run command：python ../MMDeploy/tools/deploy.py ../MMDeploy/configs/mmseg/segmentation_tensorrt_dynamic-512x1024-2048x2048.py ../mmsegmentation/configs/deeplabv3plus/deeplabv3plus_r50-d8_512x512_20k_voc12aug.py ./deeplabv3plus_r50-d8_512x512_20k_voc12aug_20200617_102323-aad58ef1.pth ../data/test.jpg --work-dir ./ --device cuda:0
2021-12-29:16:18:13,root INFO [deploy.py:73] torch2onnx start.
load checkpoint from local path: ./deeplabv3plus_r50-d8_512x512_20k_voc12aug_20200617_102323-aad58ef1.pth
/home//projects/MMDeploy/mmdeploy/codebase/mmseg/models/segmentors/encoder_decoder.py:39: TracerWarning: Converting a tensor to a Python integer might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
shape = [int(_) for _ in shape]
/home//anaconda3/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
2021-12-29:16:18:30,root INFO [deploy.py:87] torch2onnx success.
2021-12-29:16:18:30,root INFO [deploy.py:73] onnx2tensorrt of ./end2end.onnx start.
2021-12-29:16:18:34,root INFO [init_plugins.py:34] Successfully loaded tensorrt plugins from /home//projects/MMDeploy/build/lib/libmmdeploy_tensorrt_ops.so
[12/29/2021-16:18:35] [TRT] [I] [MemUsageChange] Init CUDA: CPU +323, GPU +0, now: CPU 388, GPU 332 (MiB)
[12/29/2021-16:18:35] [TRT] [I] [MemUsageSnapshot] Begin constructing builder kernel library: CPU 388 MiB, GPU 332 MiB
[12/29/2021-16:18:35] [TRT] [I] [MemUsageSnapshot] End constructing builder kernel library: CPU 522 MiB, GPU 366 MiB
[12/29/2021-16:18:36] [TRT] [W] onnx2trt_utils.cpp:366: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
[12/29/2021-16:18:36] [TRT] [W] Tensor DataType is determined at build time for tensors not marked as input or output.
[12/29/2021-16:18:40] [TRT] [W] TensorRT was linked against cuBLAS/cuBLAS LT 11.6.3 but loaded cuBLAS/cuBLAS LT 11.1.0
[12/29/2021-16:18:40] [TRT] [I] [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +354, GPU +148, now: CPU 3639, GPU 1554 (MiB)
[12/29/2021-16:18:41] [TRT] [I] [MemUsageChange] Init cuDNN: CPU +399, GPU +172, now: CPU 4038, GPU 1726 (MiB)
[12/29/2021-16:18:41] [TRT] [W] TensorRT was linked against cuDNN 8.2.1 but loaded cuDNN 8.1.1
[12/29/2021-16:18:41] [TRT] [I] Local timing cache in use. Profiling results in this builder pass will not be stored.
[12/29/2021-16:18:41] [TRT] [E] 4: [shapeCompiler.cpp::evaluateShapeChecks::832] Error Code 4: Internal Error (kOPT values for profile 0 violate shape constraints: reshape would change volume. IShuffleLayer Reshape_217: reshaping failed for tensor: 709)
2021-12-29:16:18:41,root ERROR [deploy.py:68] Failed to create TensorRT engine
Traceback (most recent call last):
File "/home//projects/MMDeploy/tools/deploy.py", line 63, in target_wrapper
result = target(args, kwargs)
File "/home//projects/MMDeploy/mmdeploy/backend/tensorrt/onnx2tensorrt.py", line 64, in onnx2tensorrt
device_id=device_id)
File "/home//projects/MMDeploy/mmdeploy/backend/tensorrt/utils.py", line 115, in create_trt_engine
assert engine is not None, 'Failed to create TensorRT engine'
AssertionError: Failed to create TensorRT engine
/home//anaconda3/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 1 leaked semaphores to clean up at shutdown
len(cache))
2021-12-29:16:18:42,root ERROR [deploy.py:84] onnx2tensorrt of ./end2end.onnx failed.
When configuring TensorRT backend for int8, both int8 and fp16 is enabled
Hello,
While looking through the code for deploying a model using TensorRT, I noticed that in the base config for tensorrt-int8, we set both FP16 and INT8=true. Is this intentional? I figured it was just a typo. I am not sure what happens when both BuilderFlags are enabled, as I have never tested it myself.
[Doc] Doc for installations
Thanks for your bug report. We appreciate it a lot.
Checklist
- I have searched related issues but cannot get the expected help.
- I have read the FAQ documentation but cannot get the expected help.
- The bug has not been fixed in the latest version.
Describe the bug
Need the links for the backends in docs:
ONNX Runtime
TensorRT
ncnn
PPLNN
OpenVINO
https://mmdeploy.readthedocs.io/zh_CN/latest/build.html#id3,
and there is no entrypoint for the installations guide: https://github.com/open-mmlab/mmdeploy/tree/master/docs/en/backends
Reproduction
- What command or script did you run?
A placeholder for the command.
- Did you make any modifications on the code or config? Did you understand what you have modified?
Environment
- Please run
python tools/check_env.pyto collect necessary environment information and paste it here. - You may add addition that may be helpful for locating the problem, such as
- How you installed PyTorch [e.g., pip, conda, source]
- Other environment variables that may be related (such as
$PATH,$LD_LIBRARY_PATH,$PYTHONPATH, etc.)
Error traceback
If applicable, paste the error trackback here.
A placeholder for trackback.
Bug fix
If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!
请问，后续是否会支持TVM？
MMOCR example from SDK
Steps completed:
- SDK Build
- Example build
While running the ocr demo in MMDeploy/build/install/example/ I'm getting this error:
./ocr: error while loading shared libraries: libmmdeploy_core.so: cannot open shared object file: No such file or directory
Can MMDeploy build in Windows10?
Because I need to deploy model with TensorRT backend on Windows10, is that possible build MMDeploy in Windows10?
Has anyone tried converting a SAR model from onnx to tensorrt format?
I have tried converting a SAR model from onnx to tensorrt format, but ended up with a failure.
I wonder where may go wrong since I have changed the config file accordingly
[TensorRT] INFO: [MemUsageSnapshot] Builder begin: CPU 4793 MiB, GPU 2718 MiB
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +747, GPU +320, now: CPU 5546, GPU 3038 (MiB)
[TensorRT] INFO: [MemUsageChange] Init cuDNN: CPU +587, GPU +266, now: CPU 6133, GPU 3304 (MiB)
[TensorRT] WARNING: Detected invalid timing cache, setup a local cache instead
[TensorRT] INFO: [MemUsageChange] Init cuBLAS/cuBLASLt: CPU +0, GPU +0, now: CPU 7026, GPU 3688 (MiB)
[TensorRT] ERROR: 1: [codeGenerator.cpp::compileGraph::476] Error Code 1: Myelin (Mismatched shape for tensor 228'ForeignNode[Cast_230___Sub_123], current shape:f32,[], vs. expected shape:f32,[1].)
2022-01-26 11:08:45,378 - mmdeploy - ERROR - Failed to create TensorRT engine
2022-01-26 11:08:45,378 - mmdeploy - ERROR - onnx2tensorrt failed.
Building mmdeploy on jetpack 4.5.1, TensorRT 7.1.3.0
Hello,
After testing mmdeploy further on a Jetson AGX platform, I have noticed a few issues when building using older tensorrt as found on jetpack 4.5.1. Unfortunately, I cannot update to latest Jetpack yet due to a custom hardware platform in use.
- Install dependencies as instructed by documentation (except h5py, as it cannot be installed on jetson)
- build MMdeploy
cd ${PROJECT_DIR}/MMDeploy
git submodule update --init --recursive
pip install -e .
mkdir build -p && cd build
cmake .. \
-DMMDEPLOY_BUILD_SDK=ON \
-DCMAKE_CXX_COMPILER=g++ \
-Dpplcv_DIR=${PROJECT_DIR}/logends/ppl.cv/cuda-build/install/lib/cmake/ppl \
-DMMDEPLOY_TARGET_DEVICES="cuda;cpu" \
-DMMDEPLOY_TARGET_BACKENDS=trt \
-DMMDEPLOY_CODEBASES=all
cmake --build . -- -j$(nproc) && cmake --install .
- Build error
...
[ 93%] Linking CXX shared library ../../../lib/libmmdeploy_segmentor.so
[ 93%] Built target mmdeploy_segmentor
[ 94%] Linking CXX shared module ../../../lib/libmmdeploy_tensorrt_ops.so
[ 94%] Built target mmdeploy_tensorrt_ops
[ 95%] Linking CXX static library ../../../lib/libmmdeploy_tensorrt_ops_static.a
[ 95%] Built target mmdeploy_tensorrt_ops_static
[ 96%] Building CXX object csrc/net/trt/CMakeFiles/mmdeploy_trt_net.dir/trt_net.cpp.o
/../MMDeploy/csrc/net/trt/trt_net.cpp: In member function ‘virtual mmdeploy::Result<void> mmdeploy::TRTNet::Init(const mmdeploy::Value&)’:
/../MMDeploy/csrc/net/trt/trt_net.cpp:153:13: error: ‘class nvinfer1::IExecutionContext’ has no member named ‘setOptimizationProfileAsync’; did you mean ‘setOptimizationProfile’?
context_->setOptimizationProfileAsync(0, static_cast<cudaStream_t>(stream_.GetNative()));
^~~~~~~~~~~~~~~~~~~~~~~~~~~
setOptimizationProfile
- Solution: In
csrc/net/trt/trt_net.cpp, replace line 153 with the following
..
#if NV_TENSORRT_MAJOR > 7
context_->setOptimizationProfileAsync(0, static_cast<cudaStream_t>(stream_.GetNative()));
#else
context_->setOptimizationProfile(0);
#endif
I am however not sure which TensorRT version introduces setOptimizationProfileAsync()
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.