Reproducing results about video-swin-transformer

[hust-nj]

Hi, before training and testing, the Kinetics400 training and validation datasets we use are preprocessed by making height of video 256, this may cause little difference. You can contact us by [email protected] for discussing more details.

[zehzhang]

Facing the same problem. I use the annotation files in this repo and test the provided pretrain models without any extra modification. Swin-T and Swin-B achieved 78.4% and 80.1% top-1 accuracy on K400, which seemed slightly worse than the reported results. @zehzhang Have you resolved this issue? If so, could you please share your solution?

Thanks for confirming the problem. I got similar decrease with SwinT (-0.4% top1 acc) and SwinB pretrained on ImageNet21k (-0.5% top1 acc). I'm reaching out to the other first co-author (referred to by @hust-nj ) and hopefully will figure out what is going on soon. I will keep this thread updated.

[hust-nj]

After a careful comparison, we find out that the performance gap is due to the slight difference on data. Our kinetics-400 data with 256 resolution was obtained (with broken video removed) from nonlocal networks which was also used in many other series of works.

More details and data download link can be found here https://github.com/youngwanLEE/VoV3D/blob/main/DATA.md#kinetics-400, facebookresearch/video-nonlocal-net#67

[dragen1860]

@zehzhang Hi, Thanks for your issues. Do you train video-swin from scratch without imagenet-21k, did it drop severely? thank you.