Lack of evaluation code and confusion on evaluation dataset about zero123

[ruoshiliu]

Hi @xyyeah , thank you for your interest in our work. I wouldn't say GSO is a subset of RTMV. A subset of RTMV, named Google Scanned Object, is composed of 300 scenes, each with 20 GSO objects. Therefore, each scene's geometry and texture is much more complicated than any single object in GSO. The experiments section of the paper as well as the appendix section have covered a lot of details regarding the evaluation process. Feel free to let me know if there's still anything unclear.

[xyyeah]

Thank you for your quick reply, I figured out the difference between GSO dataset and subset in RTMV. But I found that there are 2.5T data in the RTMV dataset, did you evaluate all the datasets or only a subset? If it is a subset, which subset did you evaluate?

[ruoshiliu]

We used 20 scenes randomly sampled from the Google Scanned Obejct subset, each scene with 17 novel views (1 for input and 16 for evaluation). Zero123 is very fast at inference time so we can easily evaluate hundreds of scenes. The difficulty in including more scenes for evaluation is that it's very expensive to train DietNeRF (one of our baseline). It takes around 24 hours to train one scene for DietNeRF on 1 GPU.

[taeyeopl]

@ruoshiliu Thanks for your explanation.

If I want to reproduce the same results of yours in Table 1 and Table 2 as you did, Can you give some information on which 20 scenes and 17 novel views are used and 1 for the input? If datasets are not large, sharing both GSO and RTMV information will be really helpful.

[xyyeah]

@ruoshiliu Thanks for your quick reply.

I would like to reproduce the same evaluation results showed in the paper. I have downloaded the GSO dataset but cannot find any paired image/camera poses. Could you share more information about the preprocess method used to obtain the test datasets for the results in Table 1 and Table 2? Or as @taeyeopl suggested, could you share the preprocessed GSO and RTMV datasets ?

[XinyangHan]

@ruoshiliu Looking for code for GSO and RTMV code as well~ It would be great if these preprocessing codes are released!!

[ruoshiliu]

Here's the script to render 25 views for GSO: gso_render.zip. I believe the RTMV dataset already contains 150 rendered views for each scene.

[XinyangHan]

Thanks so much for your help!!!

[jwlee-vcl]

@ruoshiliu Thanks for your rendering scripts.

It looks like the model names are in the test.json in your script, if you could share the test.json I would appreciate it.