x3664 / sfa3d-pointpainting

A Panoramic Awareness Network inspired by PointPainting and SFA3D

SFA3D-PointPainting

GitHub Link:A Panoramic Awareness Network inspired by PointPainting and SFA3D.

This work heavily based on two works:GitHub - maudzung/SFA3D: Super Fast and Accurate 3D Object Detection based on 3D LiDAR Point Clouds (The PyTorch implementation) and GitHub - AmrElsersy/PointPainting: Real Time Semantic Segmentation for both LIDAR & Camera using BiseNetv2 & PointPainting Fusion in Pytorch

Data Preparation

Refer to Carla-dataset-generator https://github.com/SekiroRong/Carla_dataset_generator

Installation

Run

pip install -r requirements.txt

separately in PointPainting and SFA3D folder。

PointPainting Part

Download the checkpoint

Download from Drive Place it in "BiSeNetv2/checkpoints"
Important Note The file you will download will have the name "BiseNetv2_150.pth.tar", don't unzip it .. just rename it to be "BiseNetv2_150.pth"

SFA3D Part

Visualize the dataset

To visualize 3D point clouds with 3D boxes, let's execute:

cd sfa/data_process/
python kitti_dataset.py

Inference

The pre-trained model was pushed to this repo.

python test.py

Training

Only support single GPU for now.

python train.py

SFA3D-PointPainting Part

Joint-Inference

cd SFA3D/sfa/
python joint_Inference.py

Contact

If you think this work is useful, please give me a star!
If you find any errors or have any suggestions, please contact me (Email: [email protected]) or in Issues(Preferred)

Thank you!

References

[1] CenterNet: Objects as Points paper, PyTorch Implementation
[2] RTM3D: PyTorch Implementation
[3] YOLOP：YOLOP: You Only Look Once for Panopitic Driving Perception.）
[4] PointPainting: PointPainting: Sequential Fusion for 3D Object Detection, PyTorch Implementation
[5] SFA3D：GitHub - maudzung/SFA3D: Super Fast and Accurate 3D Object Detection based on 3D LiDAR Point Clouds (The PyTorch implementation)

Citation

@misc{SFA3D-PointPainting,
  author =       {Yu Rong, Mingbo Zhao},
  title =        {{SFA3D-PointPainting}},
  howpublished = {\url{https://github.com/SekiroRong/SFA3D-PointPainting}},
  year =         {2022}
}