xubaixinxbx / 3dheads

[ICCV 2023] Code for "Deformable Model-Driven Neural Rendering for High-Fidelity 3D Reconstruction of Human Heads Under Low-View Settings"

• 👋 Hi, I’m Xu Baixin@xubaixinxbx
• 👀 I’m interested in computer vision and computer graphics, espeically neural rendering, 3d reconstruction, talking head generation
• 📫 How to reach me ...

3dheads's Issues

Custom Dataset

Thank you for sharing such great work!!

Could you please let me know, how I can reconstruct a 3D face model using custom images?

Visualization and checkpoint

Hi,

Thank you for the great work.

I found that there is no evaluation code or pre-trained model available to generate qualitative results.
Would you be able to share that?

Thank you so much

Queries Regarding the Sources for `Rt scale dict.json` and `landmark indices.npz` Files

Hello,

Firstly, I'd like to commend you on developing this fantastic project.

Currently, I'm working on training the model using the Facescape dataset. However, I've encountered a roadblock --- I'm unable to find the sources for the Rt scale dict.json and landmark indices.npz files.

Could you please guide me on how to procure these files? Your assistance in this matter would be highly appreciated.

Thank you!

code coming soon?

code coming soon

How to train with facescape datasets?

Hello, your work is very interesting. I encountered an error while trying to train using the Facescape dataset. Could you please help me answer this question.
I saw in the issue that it is necessary to add the init_hw_for_facescape function to the program, but I placed it in the position shown in the figure below and executed bash_st1.sh, resulting in the following error. Can you guide me on the correct usage of this function

how to test?

if i have some own mutl-view images, how to use the pretrained model to get 3d reconstrcution result?

Request for Facescape Dataset Training Code

Hello @xubaixinxbx,

Firstly, thank you for your work on this project. I have been trying to use the provided training code with the Facescape dataset. However, I've observed that the current training code does not seem to match with the Facescape dataset, and the results post-training are not as good as expected.

Would it be possible for you to provide training code specifically tailored for the Facescape dataset? I believe it would greatly enhance the utility of this project and help users like myself to achieve better results.

Thank you in advance for considering this request!

Best regards,

ETA for code?

ETA for code?

Zero Weights During Phase 1 Training

Hello,

I've been experimenting with your code for this project, and I've encountered an issue I'd like to discuss.

During the first phase of training, I noticed that the network's weights remain at zero throughout the process. This behavior is puzzling, and I'm not certain why it's occurring. I've attached a screenshot that shows the weights remaining at zero.

Interestingly, when I use the same FaceSpace dataset with NeuFace for training, the weights seem to be normal, indicating that the data itself is not the problem.

Could you provide some insights into this? How could we modify the network or training process to address these zero weights? Is this an expected behavior, or have I perhaps misunderstood something?

Looking forward to your advice.

Thank you,