`unable to create cuda shared memory handle` when using multiprocessing to send multiple requests about server

[Tabrizian]

Thanks for reporting this issue. I have filed an internal issue for further investigation.

[tanmayv25]

@justanhduc 22.12 is a very old release. We have made some changes in our client library to use CUDA python for cuda shared memory handle implementation.

Can you try upgrading to our latest client and let us know if you are seeing this issue here?
We are still using cudaGetDevice call so probably you would run into this issue again.

[justanhduc]

Hi @tanmayv25. Thanks for the pointer. I used the latest server docker (24.03) and the latest client and I'm still facing this issue. Could you have a look further into this issue?

[lkomali]

Hi @justanhduc
I tried to reproduce the issue and created a repro script. I used Triton 24.03. With p =1, I did not get any error and the script ran successfully.
Attaching the repro script for reference.
cuda_shm_repo.zip

[tanmayv25]

@justanhduc Can you help us reproduce the issue?

[justanhduc]

Hi guys @lkomali @tanmayv25. Sorry I didn't see the noti earlier. I will run your code and see if I can reproduce the error