julienkay / merf-unity-viewer

An unofficial Unity port of the MERF viewer

Warning !!! Work in progress !!!
Does not work, but could be fun to fix.

MERF Unity Viewer

This repository contains the source code for a Unity port of the web viewer for MERF - Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes¹

Please note, that this is an unofficial port. I am not affiliated with the original authors or their institution.

Known Issues

• Everything mostly works up to calculating the diffuse term of the volume. But the 'evaluateNetwork()' function in the shader seems to return unreasonably high values, probably due to some coordinate space or other platform-specific differences between the original Three.js viewer and Unity. Contributions welcome.

References

Other projects exploring NeRFs and related techniques in Unity:

• MobileNeRF (Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures)
• SNeRG (Sparse Neural Radiance Grids)

Footnotes

1. Christian Reiser and Richard Szeliski and Dor Verbin and Pratul P. Srinivasan and Ben Mildenhall and Andreas Geiger and Jonathan T. Barron and Peter Hedman. MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes. SIGGRAPH, 2023 ↩

merf-unity-viewer's Issues

Rendering works properly with output of torch-merf

Hi, thanks for your work!

I've trained the garden vase scene using the torch-MERF training implementation.

Surprisingly the rendering seems to work better than with the original MERF scenes that are included in this repo.

I've uploaded the trained scene on gdrive.

Functional?

Hi! Does this code work? Would love to port it to ue5.

Thanks!

Incorrect result using downloading assets and custom assets from torch-merf

Hi, Thanks for your great work.

I create a new Unity project using 2021.3.26 with built-in render pipeline, and import the package.

By downloading the Gardenvase from menu, and drag the generated prefab into an default new scene i got the following ressult:

I also trained custom data using torch-merf and import it from disk, but got similar result:

Is there any other setup needs to fix it, or is it related to the editor version.

Thank you.