how to deal with soft_scores when class_nums > 1. about sku110k_cvpr19

[eg4000]

Hi,

When you generalize to multi-class, you should first consider if you want the IoU head (as well as the standard regression head) to be class agnostic. In our experiments it worked well.
In your experiment, you have 40 different IoU scores per detection, one for each class. You can try generalizing the inference stage for class-dependent IoU score in various ways, but I recommend that you first try training a single IoU head for all classes.

As for the network you already trained, there are several possible experiments that can be made. Simple approached to reduce the 40 scores into 1 are taking the average, median, maximum, etc.

For a more elegant approaches you may consider some version of normalized weighted average of the 40 numbers, weighed according to their classification scores.

Regards,
Eran

[eg4000]

In addition, if you encounter problems with the multi-class version of the classification head, I think it would be best to look at the original code of keras-retinanet as a reference.

[djech2021]

Can anyone show me an example of how to set up a class-agnostic IoU and regression head?