Build fails: https://huggingface.co/codegen-16B-mono-hf is private about fauxpilot

[MarcSeebold]

Update: It works with "codegen-350M-mono"

[moyix]

Yep, this line is the root cause:

download_and_convert_model.sh: line 9:     8 Killed                  python3 codegen_gptj_convert.py --code_model Salesforce/${MODEL} ${MODEL}-hf

While converting the model, it loads the original model and the new model into RAM at once, so for the 16B versions that uses... a lot of RAM 😬

Doing the conversion piecemeal (e.g. layer by layer) is possible but I'd need to look a bit more deeply into the exact format of the pytorch_model.bin format.

Another option is that I could try to host preconverted versions somewhere; they're pretty big though:

1.4G    codegen-350M-mono-1gpu
1.4G    codegen-350M-mono-2gpu
1.4G    codegen-350M-multi-1gpu
1.4G    codegen-350M-multi-2gpu
11G     codegen-2B-mono-1gpu
11G     codegen-2B-mono-2gpu
11G     codegen-2B-multi-1gpu
11G     codegen-2B-multi-2gpu
27G     codegen-6B-mono-1gpu
27G     codegen-6B-mono-2gpu
27G     codegen-6B-multi-1gpu
27G     codegen-6B-multi-2gpu
60G     codegen-16B-mono-1gpu
60G     codegen-16B-mono-2gpu
60G     codegen-16B-multi-1gpu
60G     codegen-16B-multi-2gpu

I'll look into seeing what the options for doing that are.