Baize is an open-source chat model fine-tuned with LoRA. It uses 100k dialogs generated by letting ChatGPT chat with itself. We also use Alpaca's data to improve its performance. We have released 7B, 13B and 30B models. Please refer to the paper for more details.
Baize (็ฝๆณฝ) is a mythical creature in Chinese folklore, who speaks human languages and knows everything. This is exactly what we expect from a chat model.
This is the repo for the Baize project, which aims to build and share an Chat LLaMA model. This repository contains:
- 54K/57K/47K dialogs from Quora, StackOverFlow and MedQuAD questions
- The code for collecting self-chat data
- The code for fine-tuning the model
- The code for chat model demo (forked from ChuanhuChatGPT)
- Baize-7B
- Baize-13B
- Baize-30B
- Baize Healthcare-7B
- Baize Chinese-7B (Coming soon)
You can either host it on your local machine or access the online demo. The demo fetches the LLaMa model and the LoRA weights from the Hugging Face model hub, then runs a user-friendly Gradio interface for chatting.
First, make sure your Python version is 3.8, and then install the required packages using the command below:
cd demo
pip install -r requirements.txt
You can host the model on your local machine using the following command:
base_model=decapoda-research/llama-7b-hf
lora_model=project-baize/baize-lora-7B
python app.py $base_model $lora_model
Inference (without int8) | |
---|---|
Baize-7B | 16GB |
Baize-13B | 28GB |
Baize-30B | 67GB |
- Install dependencies
pip install -r requirements.txt
- If
bitsandbytes
doesn't work, install it from source. Windows users can follow these instructions.
You can use our released data or collect the data from ChatGPT using the following command:
num_process=10 # The number of processes to collect data
max_total_tokens=500000 # Set maximum numbers of tokens to collect data
api_key=xxxxxxxxxxxxxxxxx # Set your openai api key
for ((i=0; i<$num_process; i++))
do
python collect.py $api_key $max_total_tokens $i $num_process stackoverflow &
python collect.py $api_key $max_total_tokens $i $num_process quora &
python collect.py $api_key $max_total_tokens $i $num_process medical &
done
After collecting data, you use the following command to preprocess data:
python preprocess.py stackoverflow
python preprocess.py quora
python preprocess.py medical
If there's a specific dataset you want to use as seeds for ChatGPT self-chatting, you can simply modify collect.py
to load your own data.
The fine-tuning code is designed to run on an A100-80G GPU. The finetune.py
script accepts three parameters: foundation model size (i.e., 7B, 13B, or 30B), batch size, learning rate and datasets. Note the total batch size is fixed to 64 (can be modified here) and the batch size here is the per device batch size before gradient accumulation. Set it to a smaller value if you are training on a GPU with smaller VRAM.
# For the 7B model (takes about 9 hours)
python finetune.py 7b 32 0.0002 alpaca,stackoverflow,quora
# For the 13B model (takes about 16 hours)
python finetune.py 13b 16 0.0001 alpaca,stackoverflow,quora
# For the 30B model (takes about 36 hours)
python finetune.py 30b 8 0.00005 alpaca,stackoverflow,quora
With the settings above:
Training (with int8) | |
---|---|
Baize-7B | 26GB |
Baize-13B | 25GB |
Baize-30B | 42GB |
@article{xu2023baize,
title={Baize: An Open-Source Chat Model with Parameter-Efficient Tuning on Self-Chat Data},
author={Xu, Canwen and Guo, Daya and Duan, Nan and McAuley, Julian},
journal={arXiv preprint arXiv:2304.01196},
year={2023}
}