docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]] about ai-worker

[Tato14]

@yondonfu @Titan-Node I can confirm that worked with the changes mentioned earlier. To add on this, I used the Nvidia Driver 545.29.06 and Cuda version 12.3 on Ubuntu 22.
Thanks again for the great feedback!

[yondonfu]

@Tato14

• What OS are you running?
• Do you have nvidia-container-toolkit installed?

[Titan-Node]

@yondonfu I believe he is running Ubuntu 22.04

I am also getting this error error with Ubuntu 22.04.

I will try download the nvidia-container-toolkit as mentioned and try again.

[Tato14]

Yes Ubuntu 22.04

I installed the nvidia-container-toolkit and the error still persists

[yondonfu]

@Tato14 Did you also try restarting Docker and/or restarting the machine? There seems to be some other recommendations (such as this) from searching for "Error response from daemon: could not select device driver "" with capabilities: [[gpu]]" online that may be relevant depending on your configuration.

[Titan-Node]

@yondonfu @Tato14
I got the benchmark to work with the nvidia-container-toolkit installed just after I had the with capabilities: [[gpu]] issue.

I changed a few things at once.

1. Changed from Ubuntu 22 to 20 (not sure that helped)
2. Updated to Nvidia driver 551 (again not sure it helped but it was important as I tried with 535 with failure)
   At this point I still got the capabilities: [[gpu]]with capabilities: [[gpu]] error
3. Installed the toolkit including the experimental packages and configured docker and it works

[yondonfu]

@Titan-Node Glad to hear you got it working and thanks for sharing!

My guess is that step 3 (properly configuring Docker with nvidia-container-toolkit) was the important step that needed to be completed so I updated the README accordingly.