[BUG] import deepspeed, MissingCUDAException about deepspeed

[loadams]

Hi @zsaladin - the cuda runtime version and the cuda version that is bundled with torch can be different, so that is the reason why we need to be able to check the actual cuda runtime version that is installed.

[zsaladin]

@loadams Thanks for replying. I have some questions about your answer.

1. As you mentioned versions of cuda runtime installed globally and bundled with torch can be different. The cuda runtime bundled with torch is in virtual environment.
   Does deepspeed use cuda runtime in virtual environment? If deepspeed uses cuda runtime in virtual environment then the version conflict cannot happen. So it would be great that deepspeed will use bundled cuda runtime if deepspeed doesn't use it now.

2. When I use deepspeed==0.12.6 it doesn't require nvcc. nvcc is a compiler not for checking version.
   I'm not sure that but deepspeed needs to complie cuda code?

[loadams]

Hi @zsaladin - DeepSpeed uses the version of cuda runtime that is installed on the system, it cannot "use" the version that torch is built with, as that doesn't have nvcc/cuda drivers, it is just what the installed pytorch is built against.

As for why it didn't require nvcc in 0.12.6, we will have to check the code to see what changes have taken place that would cause this. With 0.12.6 does DeepSpeed detect that you are using an Nvidia GPU? Are you able to run nvidia-smi on your system with DeepSpeed 0.14.x?