whuhxb / vff

Voxel Field Fusion for 3D Object Detection (CVPR2022)

VFF

Voxel Field Fusion for 3D Object Detection

Yanwei Li, Xiaojuan Qi, Yukang Chen, Liwei Wang, Zeming Li, Jian Sun, Jiaya Jia

[arXiv] [BibTeX]

This project provides an implementation for the CVPR 2022 paper "Voxel Field Fusion for 3D Object Detection" based on OpenPCDet. VFF aims to maintain cross-modality consistency by representing and Image fusing augmented image features as a ray in the voxel field.

Installation

This project is based on OpenPCDet, which can be constructed as follows.

• Install OpenPCDet following the instructions.
• Prepare the dataset following the structure.

Training

You can train the model following the instructions. You can find the pretrained DeepLab V3 here if you want to train the model from scratch. For example, to launch PVRCNN-VFF training on multi GPUs, one should execute:

cd /path/to/vff/tools
bash scripts/dist_train.sh ${NUM_GPUS} --cfg_file cfgs/kitti_models/VFF_PVRCNN.yaml

or train with a single GPU:

python3 train.py --cfg_file cfgs/kitti_models/VFF_PVRCNN.yaml

Evaluation

You can evaluate the model following the instructions. For example, to launch PVRCNN-VFF evaluation with a pretrained checkpoint on multi GPUs, one should execute:

bash scripts/dist_test.sh ${NUM_GPUS} \
    --cfg_file cfgs/kitti_models/VFF_PVRCNN.yaml --batch_size ${BATCH_SIZE} --ckpt ${CKPT}

or evaluate with a single GPU:

python3 test.py --cfg_file cfgs/kitti_models/VFF_PVRCNN.yaml --batch_size ${BATCH_SIZE} --ckpt ${CKPT}

KITTI 3D Object Detection Results

We provide results on KITTI val set with pretrained models. All models are trained and evaluated on 8 V100 GPU.

	Car@R40	Pedestrian@R40	Cyclist@R40	download
PVRCNN-VFF	85.50	65.30	73.30	GoogleDrive
VoxelRCNN-VFF	85.72	-	-	GoogleDrive

Acknowledgement

We would like to thank the authors of OpenPCDet and CaDDN for their open-source release.

License

VFF is released under the Apache 2.0 license.

Citing VFF

Consider cite VFF in your publications if it helps your research.

@inproceedings{li2022vff,
  title={Voxel Field Fusion for 3D Object Detection},
  author={Li, Yanwei and Qi, Xiaojuan and Chen, Yukang and Wang, Liwei and Li, Zeming and Sun, Jian and Jia, Jiaya},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2022}
}