Fully differentiable training doesn't work with lower batch size about cops

[aabbas90]

Hi,

Thanks and sorry for late reply.

1. Unfortunately, this issue is because Panoptic Quality is not separable w.r.t expectation so if some class is present very less in the batch (e.g. only one person instance in the batch size of 8) and even if it has some small misclassification then it would give a huge gradient value in the loss. If you cannot afford large batch size then perhaps you can use some kind of model checkpointing to save GPU memory and still run batch size 24 instead of 8? Another option might be to apply gradient clipping here. Let me know if this still does not help.

2. Thanks for catching the edge distances issue. You are right the edge distance are indeed inconsistent. Btw the distance of 1 corresponds to 4 in full resolution i.e. these distances are computed w.r.t 4x downsampled output.