Discussion: How to tell if Nitro is using a GPU? about nitro

[tikikun]

if you are using "ngl" with value ranging from 0-3x and using a distribution of binary with cuda , it's using gpu, you can tell from the log also when model is loaded up

[Hoekz]

There seems to be a problem with the install script since version 0.2.5 where the incorrect download URL is used. There are now 2 different cuda releases:

• nitro-0.2.7-linux-amd64-cuda-11-7.tar.gz
• nitro-0.2.7-linux-amd64-cuda-12-0.tar.gz

But the download URL leaves off the -11-7 or -12-0 and only appends -cuda. We were able to navigate around this but updating the install.sh manually to include the -12-0 postfix. Either the script needs to be updated to allow for more granular selection or one of the created cuda releases needs to be assigned as a default for the script to work.

[tikikun]

i'd recommend not use install script since it's not recently updated, using pure binary file will give you the best experience

[tikikun]

@hiento09 can also have a look at the install script