some issue about "homography_converter" about cam2bev

[lreiher]

Do I correctly understand that all differences you are spotting in the computations are on the order of 1e-14 and smaller? In that case, I wouldn't worry about it, probably just floating point computation issues on that scale, which might also vary between different computers.

If currently you are only trying to reproduce the results, may I ask what MIoU values you are achieving on the provided datasets? I'm assuming that you didn't change any config files?

if I want to use my own dataset, how can I get the correct uNetXST_Homographs value?

1. Create new camera configuration files like the ones here, containing the intrinsic/extrinsic parameters of your own camera setup.
2. Run preprocessing/occlusion/occlusion.py
3. Run preprocessing/occlusion/ipm.py with your own camera config files
4. Compute uNetXST-compatible homographies with your own camera config files by following the instructions in [preprocessing/homography_converter](https://github.com/ika-rwth-aachen/Cam2BEV/blob/master/preprocessing/homography_converter
5. Adjust or create a new one-hot conversion file (model/one_hot_conversion)
6. Set all training parameters in a dedicated config file
7. Start training

[yyyzwy]

Do I correctly understand that all differences you are spotting in the computations are on the order of 1e-14 and smaller?

Yes, that's right. So maybe i don't need to worry it anymore.

If currently you are only trying to reproduce the results, may I ask what MIoU values you are achieving on the provided datasets? I'm assuming that you didn't change any config files?

Actually i am using my own dataset, and expect the the model will act just fine like it on provided dataset. But consider the difference between the dataset, this maybe be unreasonable.

I checked the images that warpped by the IPM_homographs i calculated, it seems right and make sense, so i think that the calculation of IPM_homographs about my camera settings is right. So the uNetXST_ homographies that i calculated should also be right, because it is simplely obtained by inputting IPM_homographs parameters into the conversion script.

And the MIoU of uNetXST on my dataset is about 63%, if i use the IPM image as input, send to Deeplab MobileNetV2 model, the MIoU i got is about 66%.

[lreiher]

It seems too coincidental that your own camera images would transform just fine with our dataset's camera intrinsics and extrinsics. Would you mind sharing your input images and transformed IPM images?

[yyyzwy]

No, I didn't use your camera parameters. I recalculated it for my camera settings. But the effect did not meet the expectation, so I thought maybe it was because I had miscalculated the relevant homographs parameters.

drone

IPM

front

rear

left

right

[lreiher]

Ah okay, got it. Your IPM image looks correct. If you have then converted the homographies printed by the IPM-script using the homography converter, you have correctly followed the steps.

Did you also apply the occlusion script to the drone label image?

[yyyzwy]

Yep.

[lreiher]

Okay, this is also looking good. Another thing to check might be your color palette, i.e., which color represents which class and which classes are merged before presenting input to the network.

Could you also show some predictions given by uNetXST or DL MobileNet?

[yyyzwy]

Here is a example by DL MobileNet.

The difference between input color palette and output color palette is the "Occluded" class I append in the output color palette.
And I think if there is problem with the color palette, all models should be affected.

[yyyzwy]

By the way, this is the same input result predict by uNetXST.

[lreiher]

I believe we cannot guarantee that uNetXST would always outperform the other models. I can only give one further suggestion right now what you could investigate: debugging the uNetXST model, you could take a look at the feature maps before and after the Spatial Transformer modules inside the network to check that the IPM transformation is working as expected inside the network.