float16 error in finetune.py about alltalk_tts

[erew123]

Hiya.. When you ran the diagnostics, you were in the Text-gen-webui cmd_windows.bat python environment?

The reason I ask, is its not showing you as having Pytorch with CUDA installed, so it wont find your GPU via CUDA, which is what it needs.

I don't know if this is something with the Text-gen-webui installation that occurred recently or this is just co-incidence, but your the second person Ive seen in a few days not showing CUDA.

I dont know if you installed text-gen-webui with CUDA 12.1 or 11.8....... you can force a re-install on your text-gen-webui environment by....

cmd_windows.bat in the text-gen-webui folder to start then Python environment

pip cache purge (Clear the current Pip cache)

pip uninstall torch torchaudio (Removes the current Pytorch)

You can now either install the CUDA 11.8 or 12.1 version of Torch and Torchaudio with one of the following

CUDA 11.8 > pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118

CUDA 12.1 > pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121

[davidtorcivia]

This corrected it, I don't know why pip wasn't installing these properly. Everything has been run within the text-gen-webui env (cmd_windows.bat). Might want to update diagonostic.py to ensure the cuda accelerated torch versions are present as it currently shows them as "green" in the cli even without +cu118, etc.

Just to append your commands in case someone else comes through later looking for a solution, if we're reinstalling torchvision for cu118 (or 121) then we need to run pip uninstall torch torchaudio torchvision (uninstall torchvision was missing). The install command is correct.

So the full fix is:

Enter the python environment cmd_windows.bat

pip cache purge (to rebuild clean)

pip uninstall torch torchaudio torchvision (remove old torch without cuda)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 (install with cuda 11.8, change this to pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 for cuda 12.1)

[erew123]

Hi @Jack74r

Here is a quick explainer of Python environments from the main page in the installation section:

---

A very quick understanding of Python Environments.
Different Python applications have different requirements, some of those requirement’s conflict with other Python applications requirements. To work around this problem, you can create different Python environments that remain separated from one another. A simple way of looking at Python environments, is just like how your house has different rooms for specific purposes (Kitchen, Bathroom, Bedroom etc). You can create a Python environment that is built/customised specifically for your current applications needs/purposes and will not interfere with any other Python applications environments/installations.

If you are adept at managing Python environments, have an existing Python environment and know that you won’t cause any conflicts by installing AllTalk's requirements within that Python environment, then load up your Python environment and install the requirements. For everyone else, here is a basic guide on installing AllTalk in its own custom Python environment (there will be small variations between OS's, but the principle is the same).

---

I have to admit, python environments are confusing at first.

So are you running AllTalk in a standalone installation? (I am guessing you are based on your screenshot).

From your screenshot, it doesnt look like you have started the AllTalk python environment first. I am assuming you have run the atsetup.bat file to create the custom AllTalk python environment?

Once this has been created, you would then start AllTalk with start_alltalk.bat which will load the Python environment and then AllTalk OR (for finetuning) you just want to load the Python environment, you would run start_environment.bat

When you run start_environment.bat you should see the command prompt changes, kind of similar to this (this is an example of a Python environment being loaded and you can see the way the command prompt changes, indicated by the line that the orange arrow points to)

Once you are in the Python environment for AllTalk, its now where you want to install any other requirements with PIP and also if you are finetuning e.g. pip install -r requirements_finetune.txt

You would also check that the set environment path is listing C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8\bin which you can check at the command prompt with the set command.

feel free to ask questions if you need.

Thanks

[erew123]

This was from the other person a few days ago.. also no cu118 or cu121 (Cuda 11.8 and 12.1 respectively)

[realtica]

This corrected it, I don't know why pip wasn't installing these properly. Everything has been run within the text-gen-webui env (cmd_windows.bat). Might want to update diagonostic.py to ensure the cuda accelerated torch versions are present as it currently shows them as "green" in the cli even without +cu118, etc.

Just to append your commands in case someone else comes through later looking for a solution, if we're reinstalling torchvision for cu118 (or 121) then we need to run pip uninstall torch torchaudio torchvision (uninstall torchvision was missing). The install command is correct.

So the full fix is:

Enter the python environment cmd_windows.bat

pip cache purge (to rebuild clean)

pip uninstall torch torchaudio torchvision (remove old torch without cuda)

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 (install with cuda 11.8, change this to pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121 for cuda 12.1)

Same here, thanks for the solution, I have alltalk_tts standalone, is it necessary to install Text-gen-webui?

[erew123]

Hi @realtica AllTalk will run fine without text-generation-webui and you can perform finetuning.

Thanks

[Jack74r]

Hi, thanks for your post. I've tried to follow your advice but I'm having a problem using the project for finetuning.
The installed version displays 121 so I have uninstalled and reinstalled version 118. But still the problem, and I redo the manipulation it still displays 121 but when I uninstall it displays 118 so I'm a little lost. I don't really know python.
Otherwise, everything else works fine :)

PS C:\Windows\System32> nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Sep_21_10:41:10_Pacific_Daylight_Time_2022
Cuda compilation tools, release 11.8, V11.8.89
Build cuda_11.8.r11.8/compiler.31833905_0

When i retry and run :pip install -r requirements_finetune.txt
Many installation and error show :
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
xformers 0.0.24+cu118 requires torch==2.2.0, but you have torch 2.2.1+cu118 which is incompatible.