tusen-ai / mv2d Goto Github PK

/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmdet/utils/setup_env.py:48: UserWarning: Setting MKL_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
warnings.warn(
Traceback (most recent call last):
File "tools/train.py", line 275, in
main()
File "tools/train.py", line 191, in main
cfg.dump(osp.join(cfg.work_dir, osp.basename(args.config)))
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 596, in dump
f.write(self.pretty_text)
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 508, in pretty_text
text, _ = FormatCode(text, style_config=yapf_style, verify=True)
TypeError: FormatCode() got an unexpected keyword argument 'verify'
Traceback (most recent call last):
File "tools/train.py", line 275, in
main()
File "tools/train.py", line 191, in main
cfg.dump(osp.join(cfg.work_dir, osp.basename(args.config)))
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 596, in dump
f.write(self.pretty_text)
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 508, in pretty_text
text, _ = FormatCode(text, style_config=yapf_style, verify=True)
TypeError: FormatCode() got an unexpected keyword argument 'verify'
Traceback (most recent call last):
File "tools/train.py", line 275, in
main()
File "tools/train.py", line 191, in main
Traceback (most recent call last):
File "tools/train.py", line 275, in
cfg.dump(osp.join(cfg.work_dir, osp.basename(args.config)))
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 596, in dump
main()
File "tools/train.py", line 191, in main
f.write(self.pretty_text)
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 508, in pretty_text
cfg.dump(osp.join(cfg.work_dir, osp.basename(args.config)))
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 596, in dump
text, _ = FormatCode(text, style_config=yapf_style, verify=True)
TypeError: FormatCode() got an unexpected keyword argument 'verify'
f.write(self.pretty_text)
File "/opt/conda/envs/openmmlab_MV2D2/lib/python3.8/site-packages/mmcv/utils/config.py", line 508, in pretty_text
text, _ = FormatCode(text, style_config=yapf_style, verify=True)
TypeError: FormatCode() got an unexpected keyword argument 'verify'

Hi! Thank you for your wonderful work! I reappeared the 24epoch single frame baseline but the result seems unstable,
NDS is 0.2341 and mAP is 0.1825, is this normal for this baseline? Do you have the result for this baseline? A reply will be very helpful!

Hi, thank you for sharing the great work!
I met the error below when I run the inference script.

  File "/defaultShare/SHFP12/xiaoquan.wang/01_bev/MV2D/mmdetection3d_01/mmdet3d/datasets/builder.py", line 41, in build_dataset
    dataset = build_from_cfg(cfg, DATASETS, default_args)
  File "/opt/conda/lib/python3.7/site-packages/mmcv/utils/registry.py", line 55, in build_from_cfg
    raise type(e)(f'{obj_cls.__name__}: {e}')
KeyError: "CustomNuScenesDataset: 'LoadMultiViewImageFromMultiSweepsFiles is not in the pipeline registry'"

Thank you for your great work!
Regarding the current code, I have two inquiries:
Presently, you have only executed training and testing code under the condition of batch size=1. Will there be further refinement in the future?
By what experimental facilities did you conduct the network training? How much time is required for the training of ep24 and ep72, respectively?

Hi, thanks for your great work. I find that your config image size is 512x1408, I'd like to modify to 256x704 which is common to PETR series, but the log returns all zero, I don't know where goes wrong. May you give some instructions? My training log is as follows:
mv2d_nusc_256x704.log

Hi，thanks for sharing the code，after reading the code，I wonder why you add dcn and stage_with_dcn to image backbone config, and use mask-rcnn trained in nuimage which config don't have dcn, and why you add other fpn layer to process_detector_feat, as I see in paper figure 2, you don't plot the fpn layer.

Can you please detailed the packages' version? Such as mmcv, torch, mmdet, mmdet3d, mmsegmentation, mmengine.
According to the environment version in README, I can't successfully run the code.

Thank you for your great work!

In the article, the sparse cross attention is proposed. But i don't understand how do you implement this module. It seems that you don't split the query to N groups and do n times attention operations. I think you may use cross-attn-mask and key-padding-mask to make the query attention with the selected object features. Is that true?

But using attn-mask still remains unacceptable computational complexity in attention operation and each query should calculate with all obj features. I wonder if you using a different implement or my guess is right.

Hi there, thanks for sharing the code.

I've been trying to run the evaluation using the provided checkpoint, but encountered an error in the box correlation process. Here is the error message:

MV2D/mmdet3d_plugin/models/roi_heads/utils/box_correlation.py", line 353, in epipolar_in_box
    t_rois_xymax = t_points_xymax.max(1)[0]
RuntimeError: cannot perform reduction function max on tensor with no elements because the operation does not have an identity

I was wondering if you've come across this error during your testing, and if so, do you have any suggestions on how to resolve it?
Thanks in advance.

It may be caused by using the annotation files from PETR? Do you have some customed operations on data converter on top of PETR?

I noticed that during the model forward_train, gt_bboxes_2d_to_3d is only used in the above code.
In addition, neither gt_boxxes_3d nor gt_labels_3d are subsequently used here, what is their function?

Thanks for your work. I download the MV2D-T_R50_1408x512_ep72 checkpoint, and the val result is like this. Do you have any suggestions?

At line 66 in mv2d_head.py, intrinsic[:, :2, 2] = intrinsic[:, :2, 2] - bbox[:, :2] - 0.5 / scale, but in paper,

So why do we need to divide by 0.5/scale in the formula instead of multiplying by scale？