Is GNM works for any camera? about drive-any-robot

[robodhruv]

Hi @james-yoo ,
The GNM training data certainly has a diverse variety of cameras used for data collection (and across different datasets, we did not enforce common cameras), but the data sources unanimously used cameras with a large FoV. The main motivation behind using a wide-angle camera is because the large FoV helps localization and improve distance estimates. This is generally true for all situations, and a larger FoV almost certainly improves collision avoidance as well as mitigate partial observability concerns (as opposed to something with a narrow FoV like a RealSense).

We did all our experiments with commercial fisheye cameras with two properties -- large FoV and low latency. You can also try narrower FoV and it might work, but possibly a bit worse. If you want to use a spherical camera, this might be a bit difficult since they are a lot more OOD with respect to the training data.

If you have a strong case to use other types of camera (e.g., RealSense or a spherical camera), please check out our concurrent research on using geometric priors to enable generalization to diverse camera parameters.

[james-yoo]

@PrieureDeSion,
Great. I will check ExAug. Thank you!