zerg-overmind / diff-gaussian-rasterization

diff-gaussian-rasterization

This repo contains the cuda implementation of variables for calculating Gaussian flow (both forward and backward). While the original repo of 3D Gaussian Splatting is here.

Install

git clone --recursive https://github.com/Zerg-Overmind/diff-gaussian-rasterization
pip install ./diff-gaussian-rasterization

Function

rendered_image, radii, rendered_depth, rendered_alpha, proj_means_2D, conic_2D, conic_2D_inv, gs_per_pixel, weight_per_gs_pixel, x_mu = rasterizer(
        means3D = means3D_final,
        means2D = means2D,
        shs = shs,
        colors_precomp = colors_precomp,
        opacities = opacity,
        scales = scales_final,
        rotations = rotations_final,
        cov3D_precomp = cov3D_precomp)

Where proj_means_2D are the coordinates of Gaussians in image plane, conic_2D are the inverse of 2D covariance matrices of Gaussians, conic_2D_inv are the 2D covariance matrices of Gaussians, gs_per_pixel are indices of top-K Gaussians per pixel, weight_per_gs_pixel are weights of top-K Gaussians per pixel, and x_mu are x-mu.

Feel free to change K from 20 (as in the paper) to another value by modifying here and here.

Acknowledgments: We thank the following great works DreamGaussian, DreamGaussian4D, Consistent4D, RT-4DGS, Dynamic3DGaussians, and 3DGS for their codes.

diff-gaussian-rasterization's Issues

parameter question

Hello Thanks for your excellent work.
Is proj_means_2D in your README Function the same as ["proj_2D"] in this repo https://github.com/Zerg-Overmind/GaussianFlow

Question about your rasterizer details.

Hi, Thank you for sharing your codes.
In your Readme.md , you mentioned that weight_per_gs_pixel are weights of top-K Gaussians per pixe , but in your cuda implementation. https://github.com/Zerg-Overmind/diff-gaussian-rasterization/blob/d5f00e7d8a4757a06595805259323b9924383ac6/cuda_rasterizer/forward.cu#L389 it seems that what you return is the cumulative wetight instead of weight per gaussian. And in your paper, you normalize the weights by dividing their sums. Is this a bug or i misunderstood your meaning?

Regards.

Potential bug in forward.cu

Hi @Zerg-Overmind, thanks for open sourcing this great repo.

In forward.cu#L391, the y coordinate of x_mu seems to be indexed incorrectly.

x_mu[calc * H * W * 2 + pix_id] = d.y; 

Assuming that top K=20 Gaussians, it seems that it should be changed to the following:

x_mu[20 * H * W + calc * H * W + pix_id] = d.y;

These are my thoughts at the moment. Please forgive me if I'm mistaken.

About rendered_depth

May I ask whether your rendered_depth can be used for some tasks with depth supervision, or can it only be visualized?

Dimension check of the 2D cov and invese

Hi thanks for the great work! I just have a question about the dimensions of the output 2D covariance and its inverse.
In the pseudo-code, the dimension of the covariance was Nx2x2 but here the output was Nx3.
Did I miss anything? Or are there some other operations not added?

Thanks!