mps256 / vicuna.ps1

This is a PowerShell script that automates the process of setting up and running VICUNA on a CPU (without a graphics card) using the llama.cpp library and a pre-trained ggml-vicuna-13b-4bit.bin model. The script downloads and extracts the required files, creates a batch file to run VICUNA, and creates a desktop shortcut to launch the batch file.

Automatic installation script for Vicuna (CPU-only)

Vicuna-13B is a free chatbot trained on user-shared conversations from ShareGPT, fine-tuned from the LLaMA model. It outperformed other models like OpenAI ChatGPT, Google Bard, LLaMA, and Stanford Alpaca in more than 90% of cases.

• This is a PowerShell script that automates the process of setting up and running VICUNA on a CPU (without a graphics card) using the llama.cpp library and a pre-trained GGML Vicuna model. The script downloads and extracts the required files, creates a batch file to run VICUNA, and creates a desktop shortcut to launch the batch file.

• Linux and macOS scripts are currently being developed and will be available in the near future.

Requirements

• Windows operating system
• PowerShell

Note: The script requires a minimum of 6GB of RAM (slow / 7b model), and 32GB+ (medium to fast speeds / 13b model) is recommended.

Automatic installation (Recommended)

• To automatically install the program, follow these steps (Right-Click your mouse to paste the commands):

1. Press Windows + R, type powershell, and hit enter.
2. Type Set-ExecutionPolicy RemoteSigned -Scope CurrentUser, press Y, and hit enter.
3. Type irm bit.ly/vicuna_ps1 | iex and hit enter.

Manual installation (Optional)

1. Download the vicuna.ps1 script to your computer.
2. Open PowerShell and navigate to the directory where the script is saved.
3. Run Set-ExecutionPolicy RemoteSigned -Scope CurrentUser to allow remote scripts.
4. Run the script using the command .\vicuna.ps1.
5. Follow the on-screen instructions to complete the setup process.
6. Once setup is complete, double-click the "VICUNA" shortcut on your desktop to run VICUNA.

speed = 2 * log2(ram_gb / 10) + 1

        |       Speed
        |   Slow   Medium    Fast 
     -----------------------------
        |         
    3   |          
        |          
    2   |        * 
        |    *     
    1   | *        
        |_____________________________
             10GB      32GB+       RAM

• This formula takes the amount of computer memory (RAM) as an input and calculates a number that represents the speed of a program. The more RAM you have, the faster the program will run.

• ChatGPT 4 note below:

Planned Features

• Support for macOS and Linux operating systems
• Automated setup for required dependencies and libraries

Troubleshooting

If you encounter any issues while running the script, please check the following:

• Ensure that your computer meets the minimum RAM requirements.
• Check that PowerShell is running as an administrator.
• Ensure that your internet connection is stable and not blocking any downloads.
• Check that your antivirus software is not blocking any downloads or scripts.

If you continue to experience issues, please contact the developer for assistance.

Credits

This is a joint effort with collaborators from multiple institutions, including UC Berkeley, CMU, Stanford, UC San Diego, and MBZUAI.

Students (alphabetical order): Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang

Advisors (alphabetical order): Joseph E. Gonzalez, Ion Stoica, Eric P. Xing LMSYS Special thanks to eachadea (Chad Ea-Nasir II) for their 4-bit quantized model and SpreadSheetWarrior for their tutorial on running VICUNA on CPU.

Acknowledgment

We would like to thank Xinyang Geng, Hao Liu, and Eric Wallace from BAIR; Xuecheng Li, and Tianyi Zhang from Stanford Alpaca team for their insightful discussion and feedback. BAIR will have another blog post soon for a concurrent effort on their chatbot, Koala.

vicuna.ps1's Issues

7B Auto Vicuna

I'm sure you are working on it already. Just want to encourage you with this issue!
https://huggingface.co/eachadea/ggml-vicuna-7b-4bit

Thank you for making this!

Linux Support

Will this have Linux support soon?

we need an autoVicuna using the same autoGPT design but using vicuna instead of OpenAI

That would be simply great!

Press any key to continue loop?

Why am I stuck in a dead-end cycle?

like this :"
mian : seed = 1681381082
press any key to continue...
"

Launch error

Failed to open logfile 'main.log' with error 'Permission denied'
[1698425118] Log start
[1698425118] Cmd: C:\VICUNA\llama.cpp\main.exe -i --interactive-first -r "### Human:" -t 8 --temp 0 --color -c 2048 -n -1 --ignore-eos --repeat_penalty 1.2 --instruct -m C:\VICUNA\ggml-vicuna-13b-4bit.bin
[1698425118] main: build = 1431 (c8d6a1f)
[1698425118] main: built with MSVC 19.35.32217.1 for x64
[1698425118] main: seed = 1698425118
[1698425118] main: llama backend init
[1698425118] main: load the model and apply lora adapter, if any
gguf_init_from_file: invalid magic characters tjgg.
error loading model: llama_model_loader: failed to load model from C:\VICUNA\ggml-vicuna-13b-4bit.bin

llama_load_model_from_file: failed to load model
llama_init_from_gpt_params: error: failed to load model 'C:\VICUNA\ggml-vicuna-13b-4bit.bin'
[1698425118] main: error: unable to load model
Press any key to continue . . .

Wrong path

On PC PowerShell script placed main.exe into folder C:\VICUNA\llama-master-master-cc9cee8--x64\main.exe but vicuna.bat is expected at C:\VICUNA\main.exe