About WSconv2d about nfnets_pytorch

[benjs]

You are right, on first sight, the Stem layers need an additional padding='1' (all convolutions with kernel size 3).
I will take a closer look soon.

It is constant, because I did not implement something fancier yet. What do you think it should be?

Thank you!

[fancyshun]

You are right, on first sight, the Stem layers need an additional padding='1' (all convolutions with kernel size 3).
I will take a closer look soon.

It is constant, because I did not implement something fancier yet. What do you think it should be?

Thank you!

when kernel size is 3, padding should be different, because stride is 1 or 2.
I see the author's code learning rate is changed by warm up cosine decay.
I also want to know if I train with multiple gpus, how to change batch size.

[benjs]

This repo does not support multi gpu training, you can only reuse the model code for that.