YF doesn't work for the cv task about yellowfin_pytorch

[JianGoForIt]

Hi @EdwardTyantov,

Thanks for trying out our optimizer.

Seen from the figure you provided, I think there are two things you want to try out.

1. There are exploding or zero gradient in the middle of training, which drive the learning rate and momentum changing crazily. You might want to do gradient clipping to avoid this, Please refer to the solutions in the discussion here #1

2. It seems there is an initial increasing of training loss, it might help if you play with the initial learning rate a bit to eliminate initial increasing of the training loss.

Hope it helps and please let us know whether it helps or not.

[EdwardTyantov]

Thank you for the quick reply!

1. OK. I'll try clipping, will report on the results here )
2. I've already played with it - wide range on which SGD showed acceptable performance.

[jtoy]

@EdwardTyantov please share your findings here!

[JianGoForIt]

Closed due to lack of activity. Feel free to reopen if necessary.