Comments (1)
Thanks for the questions!
- Mainly for performance reason. In our experiments (Sec 5.3), our depth-peeling implementation could be 10-20x faster than the cuda-based implementation. We also tested Pulsar and it's also slower than our impl. A comparison was also made with 3DGS and similar speeds are obtained (with many other advantages from other components (App. B.3)). My personal thought on this is that the hardware accelerated rasterization pipeline is reasonably fast (faster than CUDA-based software rasterization) when the number of points is reasonable (250K). Switching from depth-peeling to 3DGS might make our method scale better to extremely large number of points, but we didn't observe the need for such large point count in our experiments.
- We could (and did) use other initialization techniques for generating the initial point clouds. In the paper, when foreground masks are not available (like when modeling the static background), we trained an Instant-NGP model, as stated in Sec. 4. Space carving was a consistently working and good enough choice for our setting.
- I do think this is an interesting point for improving convergence (although this will limit the setting a little).
- Do you mean applying temporal priors like optical flow? I do think this might work and make training easier (with the benefit of producing correspondence). Although in our early experiments we find optical flow estimators not consistent and reliable enough when the motion is fast and extremely complex. This could improve the initial training but might limit the final quality.
- It does handle the view dependency present in the experimented dynamic multi-view dataset well enough (as shown in the videos on the project page). IBR + SH provides a quite robust view-dependency model for our 4D view synthesis model.
- For now, we're focusing on making 4K4D able to handle longer videos and produce correspondences.
from 4k4d.
Related Issues (20)
- Are there any detailed steps for training our own data? HOT 10
- error in imgui_bundle HOT 5
- version `GLIBC_2.33' not found HOT 2
- Extraction of depth map and normal map HOT 1
- training speed HOT 1
- Runtime exception: Could not find {optimized/intri.yml,optimized/extri.yml} HOT 1
- There was a problem rendering the smallest data set HOT 1
- Bad result on custom dataset HOT 5
- Render my trained model HOT 1
- Training Error: launcher() got an unexpected keyword argument 'viewer_cfg' HOT 2
- error in evc-gui -c configs/projects/realtime4dv/rendering/4k4d_0008_01.yaml,configs/specs/vf0.yaml HOT 2
- error in python scripts/realtime4dv/extract_images.py
- error in: evc-train HOT 2
- An error occurred during the joint training model conversion. HOT 3
- Difference From 4DGS HOT 6
- Got bad rendering quality than expected after training 4K4D on NHR dataset HOT 4
- Question about the pretrained models HOT 1
- Evaluation: rendering the test views HOT 3
- Training on NHR HOT 1
- eglQueryDeviceAttribEXT is not available.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from 4k4d.