The Pytorch implementation of TSGD algorithm in:'Scaling transition from SGDM to plain SGD'
https://arxiv.org/abs/2106.06749
The implementation is highly based on projects AdaBound , Adam , pytorch-cifar, thanks pretty work.
The test environment we passed is: PyTorch=1.7.0, Python=3.7.10, Linux/Centos8.3.
Please directly download the TSGD folder and put it in your project, then
from tsgd import TSGD
...
optimizer = TSGD(model.parameters(), iters=required, lr=required)
#iters(int, required): iterations
# iters = math.ceil(test_size / batch_size) * epochs
The code will be uploaded as soon as possible.