Socket timeouts in MII about deepspeed-mii

[mrwyattii]

@mayank31398 thanks for reporting this. Could you share what model / MII config you are using? I'd like to reproduce this error on my side. Thanks

[mayank31398]

@mrwyattii this was a bug in my code and not in MII.
Closing this for now :)

[mallorbc]

@mayank31398 What was the cause of the issue? What was the fix? I am having this issue myself with trying to run this with FastAPI

[mayank31398]

@mallorbc, I personally never managed to fix it. So, I re-implemented some of the stuff in MII.
If you are trying a NLP task, you can use https://github.com/huggingface/transformers-bloom-inference
Let me know if it helps.