keksboter / web-splat

3D Gaussian Splatting Renderer implemented in WebGPU (WGPU) and Rust

3D Gaussian Splatting Renderer implemented in WebGPU (WGPU) and Rust

This code implements a renderer for the reconstructions obtained with 3D Gaussian Splatting

👉 Click to run the web demo 👈

Build

First, install the Rust compiler (e.g. with rustup)

Clone the repository and run

cargo build --release --bin viewer 

Run

Use the point_cloud.ply and cameras.json files generated by 3D Gaussian Splatting:

cargo run --release --bin viewer point_cloud.ply cameras.json

To load compressed npz files the npz feature must be enabled:

cargo run --release --features npz --bin viewer point_cloud.npz cameras.json

Usage

3D Gaussian Splatting Viewer

Usage: viewer [OPTIONS] <INPUT> [SCENE]

Arguments:
  <INPUT>  Input file
  [SCENE]  Scene json file

Options:
      --no-vsync  
  -h, --help      Print help
  -V, --version   Print version

About

Splat Sorting : We ported the Fuchsia RadixSort to WGPU for sorting the splats on the GPU.

Performance: The renderer reaches >200 FPS on a NVIDIA 3090 RTX and ~130 FPS on a AMD Radeon R9 380 Series (8 years old). Measurements where taken for the bonsai scene at 1200x799 resolution.

Citation

If you find our work useful, please cite:

@misc{niedermayr2023compressed,
    title={Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis}, 
    author={Simon Niedermayr and Josef Stumpfegger and Rüdiger Westermann},
    year={2023},
    eprint={2401.02436},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

web-splat's Issues

How to create a href?

Dear:
I'm a undergraduate from China,recently,my mates and I want to create a href like the following one for our own point_clouds.ply and cameras.json files?

href="https://keksboter.github.io/web-splat?file=./scenes/bicycle/point_cloud/iteration_35000/ point_cloud.npz&scene=./scenes/bicycle/cameras.json">

We want to show this following scene in chrome instead of showing by cargo run --release --bin viewer point_cloud.ply cameras.json
Could you give us some suggestions.

bb1609afd1bd63e1d37f28d144be818d.mp4

                                                                                                                                                                                             Sincerely.

build viewer failed

Thanks for your great works!
I get the following error when I run the command "cargo build --release --bin viewer ".

I'm not familiar with rust. Can you give me some advice?

Aspect ratio is not maintained if display window is resized

Running on Ubuntu 23.04, on a custom dataset, the aspect ratio of the physical object (square) is not maintained on window resize.

But if the window's aspect ratio is set to exactly to that in cameras.json (width and height), the physical object looks fine. This can be seen in the attached dataset when the aspect ratio is drastically changed.

Looking at the code, I do see that the CameraUniforms/projection matrix is changed on a window resize. Not sure why the aspect ratio is not maintained though. Please let me know if I need to do any special initialization.

Viewer freezes on M2 Pro

Settings

• Apple M2 Pro
• Google Chrome 123.0.6312.87 (arm64)
• Metal 3

Descriptions

Freezes at or after unpacking some sample models. I don't know why and can't record this (My whole screen freezes). The memory usage looked normal. My cursor can move. However, my computer just shutdown unexpectedly.

Expectations

It should work on Google Chrome (arm64) and on Apple's Metal 3. Since it is based on WebGPU.

How to solve the problem of having 20M + points

Thank you for the great work. I'm trying your work on compressing my large-scale scene, everything works great until I try to visualize it using the viewer. It seems like there are too many points to be rendered using the current implementation. specifically, the error/limitation occurred as:
Caused by:
In a ComputePass
note: encoder = render command encoder
In a dispatch command, indirect:false
note: compute pipeline = preprocess pipeline
Each current dispatch group size dimension ([153754, 1, 1]) must be less or equal to 65535

Which originated from render.rs line 409:
let wgs_x = (pc.num_points() as f32 / 256.0).ceil() as u32;
pass.dispatch_workgroups(wgs_x, 1, 1);

Is there any work around for this to handle more points in the viewer?

Thank you in advance for the help!

Missings arguments in 'open_window' when compiling to targetArch "wasm32"

Hi Simon,
I've encountered an issue while attempting to compile the web-splat renderer into WebAssembly. The following errors were encountered:

error[E0061]: this function takes 5 arguments but 3 arguments were supplied
--> src/lib.rs:769:39
|
769 | wasm_bindgen_futures::spawn_local(open_window(
| __________________________________^^^^^^^^^^^-
770 | | pc_reader,
771 | | scene_reader,
772 | | RenderConfig { no_vsync: false,skybox:None },
773 | | ));
| |- two arguments of type std::option::Option<PathBuf> and std::option::Option<PathBuf> are missing
|
note: function defined here
--> src/lib.rs:554:14
|
554 | pub async fn open_window<R: Read + Seek + Send + Sync + 'static>(
| ^^^^^^^^^^^
555 | file: R,
| -------
556 | scene_file: Option,
| ---------------------
557 | config: RenderConfig,
| --------------------
558 | pointcloud_file_path: Option,
| -------------------------------------
559 | scene_file_path: Option,
| --------------------------------
help: provide the arguments
|
769 | wasm_bindgen_futures::spawn_local(open_window(pc_reader, scene_reader, RenderConfig { no_vsync: false,skybox:None }, /* std::option::Option /, / std::option::Option */));
|

How to build the web viewer?

Thanks for sharing the great works!

I have successfully ran the viewer.exe in windows, but failed in linux. The problem is that the linux server don't have a GUI, and I got the following output:

thread 'main' panicked at /data/c3dgs/web-splat/src/lib.rs:499:39:
called Result::unwrap() on an Err value: Os(OsError { line: 776, file: "/data/.cargo/registry/src/index.crates.io-6f17d22bba15001f/winit-0.29.10/src/platform_impl/linux/mod.rs", error: Misc("neither WAYLAND_DISPLAY nor DISPLAY is set.") })
note: run with RUST_BACKTRACE=1 environment variable to display a backtrace

I know nothing about the rust, but it looks like the viewer can run in a web viwer.
Can you please kindly provide some detials of building the web viewer?

M1 Poor Performance

Its pretty much unusable on my M1, is this meant for more performant systems that have nvidia/amd gpus?