billschumacher / auto-vicuna Goto Github PK

would be great if there was a colab notebook to run vicuna.
we can load weights from huggingface model hub and test it out on colab

Hi, I am trying to follow the notebook InteractiveManual.ipynb

I have downloaded both llama-13b-hf and vicuna-13b-delta-v1.1 locally.
I am using fschat-0.2.11 and transformers-4.28.1
GPU 2*48GB VRAM

While running this,

convert_llama_names("./decapoda-research/llama-13b-hf") !python -m fastchat.model.apply_delta --base ./decapoda-research/llama-13b-hf --target ./lmsys/vicuna-13b --delta ./lmsys/vicuna-13b-delta-v1.1

I am getting the following error while Loading checkpoint shards:
NotImplementedError: Cannot copy out of meta tensor; no data!

Any idea why this is happening?
Thanks for reading this.

Can't figure it out, need help.

Can I run this with AutoGPT that uses Vicuna and can browse the web?

First of all, thank you for your work. However, I encountered some errors when using the colab jupyter script. It might be that you have not updated the script. Could you please update the latest script if it is convenient for you?

on the file main.py they ask for device, conversation_from_template("vicuna_v1.1"),
in fastchat.conversation.py you find this

default_conversation = conv_v1_2
conv_templates = {
"v1": conv_v1_2,
"bair_v1": conv_bair_v1,
}

main.py (row 90)
search for device, conversation_from_template("vicuna_v1.1"),
change from "vicuna_v1.1" to "v1"
its looks like device, conversation_from_template("v1"),