service "triton" refers to undefined volume about fauxpilot

[Frederisk]

Always use launch.sh to start containers, this file contains commands to set these environment variables.

If you want to avoid executing this script afterwards, and just use docker compose up one directly. You can

• replace ${GPUS} as the comma-separated array starting with zero with the same number of GPUs in your device, and

  For example, if there is 1 then it is 0, and if there are 2 it is 0,1.

• replace ${MODEL_DIR} as the path where model is located, but need to go one level deeper.

  For example, if your model is on path /home/user/fauxpilot/model, the used model is codegen-6B-multi, and you device has 1 GPU, then here should be like /home/user/fauxpilot/model/codegen-6B-multi-1gpu.

Prepending ${MODEL_DIR} to
./ is wrong, because in scripts there will usually be an absolute path.

[elmotec]

Ah! I see: I did use launch.sh at first. I probably set MODEL_DIR incorrectly in the setup.sh script and it went downhill from there:

Like this:

~/dev/fauxpilot$ ./setup.sh
Models available:
[1] codegen-350M-mono (2GB total VRAM required; Python-only)
[2] codegen-350M-multi (2GB total VRAM required; multi-language)
[3] codegen-2B-mono (7GB total VRAM required; Python-only)
[4] codegen-2B-multi (7GB total VRAM required; multi-language)
[5] codegen-6B-mono (13GB total VRAM required; Python-only)
[6] codegen-6B-multi (13GB total VRAM required; multi-language)
[7] codegen-16B-mono (32GB total VRAM required; Python-only)
[8] codegen-16B-multi (32GB total VRAM required; multi-language)
Enter your choice [6]: 1
Enter number of GPUs [1]:
Where do you want to save the model [/home/username/dev/path/to/models]? models
Downloading the model from HuggingFace, this will take a while...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   287  100   287    0     0   2023      0 --:--:-- --:--:-- --:--:--  2035
100  803M  100  803M    0     0  40.8M      0  0:00:19  0:00:19 --:--:-- 38.4M
Done! Now run ./launch.sh to start the FauxPilot server.
~/dev/fauxpilot$ ./launch.sh
ERROR: Named volume "models/codegen-350M-mono-1gpu:/model:rw" is used in service "triton" but no declaration was found in the volumes section.
~/dev/fauxpilot$ cat config.env
MODEL=codegen-350M-mono
NUM_GPUS=1
MODEL_DIR=models

In that case, maybe a conversion to absolute path would make setup.sh fool proof.

index 0a8e35b..f10d477 100755
--- a/setup.sh
+++ b/setup.sh
@@ -40,6 +40,7 @@ read -p "Where do you want to save the model [$(pwd)/models]? " MODEL_DIR
 if [ -z "$MODEL_DIR" ]; then
     MODEL_DIR="$(pwd)/models"
 fi
+MODEL_DIR=$(cd $MODEL_DIR && pwd) || exit $?

 # Write config.env
 echo "MODEL=${MODEL}" > config.env

[Frederisk]

Good idea, but the cd command requires that the path after that must already exist, which is not guaranteed. Perhaps explicitly asking the user to enter an absolute path here is a better option?

[moyix]

I think we could also use readlink -f or realpath to get an absolute path to the model directory based on what the user inputs in setup.sh, and that should fix this pretty comprehensively.

[Frederisk]

Or -m should be used to avoid problems for users when entering multi-components like foo/foo/foo.🤔

[moyix]

I think that PR fixes this issue, I'll close it now :)

[elmotec]

Yes, thanks both.