Different result in disjoint 15-5s setting. about mib

[fcdl94]

Hey! Probably you get different results due to a different batch size...
This setting is particularly challenging due to the non-i.i.d. data, so probably decreasing the BS hampers the performances.

[kona419]

Hey! Probably you get different results due to a different batch size... This setting is particularly challenging due to the non-i.i.d. data, so probably decreasing the BS hampers the performances.

Thank you for the reply.
Because of my GPU memory, my batch size has a limit.
So could you recommend other hyperparameters(ex. learning rate, weight decay) for the low batch size to follow the paper's result?

[fcdl94]

I actually never tried with a lower batch size. The main issue is using a low batch size in the 15-1 increasing the non i.i.d.-ness of data (you may try to use Batch Renormalization in place of BN as in my https://arxiv.org/abs/2012.01415, but it may alter - in positive - the results).

As a rule of thumb, you may double the iteration and halve the learning rate but I won't guarantee it will work.

[kona419]

I actually never tried with a lower batch size. The main issue is using a low batch size in the 15-1 increasing the non i.i.d.-ness of data (you may try to use Batch Renormalization in place of BN as in my https://arxiv.org/abs/2012.01415, but it may alter - in positive - the results).

As a rule of thumb, you may double the iteration and halve the learning rate but I won't guarantee it will work.

Thank you for sharing~!
I will try with your recommendation.