Training cannot start about synchronized-batchnorm-pytorch

[shuuchen]

I wrote a test script which solved the hanging problem. I guess there is something wrong with reduce function.

Please refer to the script:
https://gist.github.com/shuuchen/7463009370e9ddf77e649f3fec259024

You can adapt the code with your own task easily.

[AManHasNoName12138]

@shuuchen I met the same problem. Have you solved it?

[rvandeghen]

Up ?

[rvandeghen]

@shuuchen I guess you used the pytorch tutorial on DDP. I've dived into it and will most likely use the SyncBatchNorm implemented by pytorch as well.
Anyway thank you as it may help others :)

[zwyking]

I wrote a test script which solved the hanging problem. I guess there is something wrong with reduce function.

Please refer to the script:
https://gist.github.com/shuuchen/7463009370e9ddf77e649f3fec259024

You can adapt the code with your own task easily.

have you find out the exact problem in this provided syncBatchNorm?

[zwyking]

@shuuchen I met the same problem. Have you solved it?

hi, have you solved this problem?

[vacancy]

See also #44 (comment)