A few questions about your work about cam2bev

[lreiher]

1. As stated in the paper, the synthetic training data was generated with a simulation tool called Virtual Test Drive (VTD). The ground truth therefore is directly obtained from the simulation. The semantic class coloring palette is akin to Cityscapes.
2. Yes, that's correct. Note however that the results in the paper were obtained with perfectly segmented input images from simulation. In practice, you would have to run a dedicated segmentation model first, as we also did in section V.B.
3. Yes, in III.D, IPM is basically integrated into the network. The input to the network is still semantically segmented images in order to decrease the domain gap between synthetic training data and real-world data.
4. You need to have ground truth data at hand, i.e. semantic segmentation in BEV. Our approach is to generate synthetic training data using simulation tools such as VTD or CARLA, giving us the ground truth in BEV basically for free. By running on semantically segmented input images, we hope to decrease the domain gap between simulation and real world, s.t. we can successfully apply a trained model in the real world as well. In your specific case, you might also want to consider generating synthetic datasets or alternatively you could first of all see whether standard IPM already gives you usable results (see our ipm.py).