vdigpku / drivinggaussian

[CVPR 2024] DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes

DrivingGaussian👋

This is the official implementation of DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes.

Paper | Xiaoyu Zhou📧, Zhiwei Lin📧, Xiaojun Shan📧, Yongtao Wang📧, Deqing Sun📧, Ming-Hsuan Yang📧

Update

• 2024/05/10 - Code: please sign the application to obtain the code
• 2024/03/13 - Pre-trained weights are released
• 2024/02/27 - Paper: Accepted by CVPR 2024 👏
• 2023/12/07 - Webpage

Weights

Pre-trained weights for certain scenes are released: Google Cloud

Requirements

Setups

Please follow the 3DGS to install the relative packages.

git clone https://github.com/VDIGPKU/DrivingGaussian
cd DrivingGaussian
git submodule update --init --recursive
conda create -n DrivingGaussian python=3.8 
conda activate DrivingGaussian

pip install -r requirements.txt
pip install -e submodules/depth-diff-gaussian-rasterization
pip install -e submodules/simple-knn

Training

For training the driving scenes dataset,

python train.py -s "data/nuscenes/sceneID/" -m "saved/checkpoints/"

Rendering

Render the images with the following script

python render_combine.py -m "saved/checkpoints/"

Tools and Preprocessing

Acknowledgements

The overall code and renderer are based on 3DGS and 4DGS. We sincerely thank the authors for their great work.

License

The project is only free for academic research purposes, but needs authorization forcommerce. For commerce permission, please contact [email protected].

drivinggaussian's Issues

When will the code be released？

Is there anyone obtain the code?

Congratulations on being accepted by CVPR2024. When will release the code ？

Question about Incremental Gaussian Fusing and Optimization

Thank you for your great work and sharing it! While studying your paper I was a little confused about how the 3D gaussians from different bins are created and optimized.

I am specifically confused about: "For the subsequent bins, we use the Gaussians from the previous bin as the position priors [...]". Could you please explain:

• How are the previous gaussians used as position priors? I thought the LiDAR pointcloud was used as initialization
• Do you also divide the whole LiDAR pointcloud into bins and use the separate pointcloud chunks to initialize new gaussians in every bin?

Another related doubt I had is regarding the optimization. As far as I can understand from Figure 3 (see below) and Figure 8, the whole map is being kept in GPU memory and optimized jointly even as new bins are added to the scene.

• Could you please confirm whether that is the case?
• Do you have any specific adjustments to the optimization strategy as the camera moves further and further from the initial bin?

Thank you in advance for your help!

what scenes are used in the NuScenes？

Hello, I would like to ask what scenes are used in the NuScenes in the paper? Is it convenient to make it public first?