How to chose total number of training steps about mixmatch

[david-berthelot]

This methodology is not MixMatch specific, we used it for all methods. Its purpose was not to stop overfitting but to minimize variance of measurements.
Generally we found the longer you train the better, we trained for 2^16K images (2^26 images) for time constraints (that's roughly a 1000 epochs).
Parameters and settings in our experiments seemed robust across all datasets.
The best suggestion I can offer is to experiment on your data, there's no secret recipe.

[hzhz2020]

thanks a lot for your response and advice.

[hzhz2020]

Hello again

regarding your previous reply. I am a little confused do you mean 2^16K images is an epoch? If so, why is that (if we think of an epoch as going through the whole dataset once)?

'we trained for 2^16K images (2^26 images) for the time constraints (that's roughly a 1000 epochs)'.

Thank you!

[david-berthelot]

cifar10 has 50K images, 2^26/50000 = 1342 the exact number of training epochs.
However since almost every dataset has a different number of samples, to keep the code simple we decide to call an epoch 65536 images (2^16) and share that number for all datasets, so that every experiment on every dataset is trained on the same number of images.

[hzhz2020]

Thanks for clear things up!