Test set contaminated with Trainval objects! about ai-tod

[Chasel-Tsui]

Hi Alex,

Thanks for your interest in the AI-TOD series. I think this observation is interesting, and we haven't noticed it yet. Given that the original version of this dataset has been used in the community for benchmarking algorithms for more than two years, it is hard to make adjustments to it and filter out overlapping parts if they do exist.

In this case, I recommend using v2 since the number of overlapping boxes takes up a small portion of the whole dataset. Besides, as long as the method was evaluated under the same experimental setting, I think this dataset is still a good candidate for benchmarking tiny object detection since the sota mAP is still extremely low. Moreover, you can filter out these overlapping boxes and get a high-quality dataset if you would like to.