magicleap / deltas Goto Github PK

Hi, thanks for your excellent work. In the original paper you write:

3.4 Overall training objective
(b) a smooth-L1 loss between the 2D points output after soft argmax and ground truth 2D point matches...

I am a bit confused by the term "ground truth 2D point matches". Are they obtained by transforming the 2D points in a reference image to a target image using the ground truth pose?

Hello!
I found that the evaluation results of DPSNET on SUN3D in your paper have obvious differences with the original paper. I check your code, and I found that you downsample the GT depth to (320,240) to evaluate with pred_depth while prior works usually upsample the pred_depth to the original size to do the evaluation. Although this minor difference won't influence your paper's conclusion, maybe you could clarify this difference in the READE.ME to prevent other readers from misusing your numbers reported in your paper. If possible, may you report your tables with the original size like other works and readers could easily cite your tables?

Thank you.

Hello!
First of all, DELTAS is a great research and an easy-to-work-with code, so thank you.
I see in the paper that you report the forward pass time as ~90 milliseconds on a Titan RTX. I am using image size 320 x 240 for all of the upcoming values that I write here. Also I am using the faster SVD implementation that you share. I am trying to measure the inference time on a GTX 1060 and a GTX 1080Ti, but I can only measure ~530 milliseconds on 1060, and ~500 milliseconds on 1080Ti per prediction with batch size = 1. If I increase the batch size to 8, I can get around 200 milliseconds on 1080Ti per prediction. It might be possible to get 90 milliseconds per prediction with batch size 16 on a Titan RTX, unfortunately I can not test it. But I am not sure if it is meaningful to measure the inference time with batch size 16 anyway, in fact anything over 1 is pretty much always memory dependent and does not reflect the real frames per second value. The problem is, I am able to measure ~90 ms for MVDepthNet (this is close to what you report as ~60 ms), and ~380 ms for DPSNet with batch size = 1. And you clearly state that DPSNet is 2.5 times slower than DELTAS.
Maybe I am making a mistake, could you please share the code or share some details on how you measure the inference time?

hey could u tell any insights on genrating depth png images like scannet ?

Thank you for your remarkable work!

The implementation of learnable triangulation is a great inspiration.

But I'm confused about the function "patch_for_depth_guided_range" in triangulation.py. The calculation of the fundamental matrices and epipolar lines make sense to me. But the meaning of variables like x2ord, y2ord, x2, and y2 isn't clear to me.

Would you please help to explain this function? Thank you.