tokenizer.pad_token_id = 0

Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████| 39/39 [00:11<00:00,  3.38it/s]
Some weights of the model checkpoint at ../alpaca-lora/hf_ckpt/ were not used when initializing LlamaForCausalLM: ['base_model.model.lm_head.weight']

(.venv) [danielw@pc GPTQ-for-LLaMa]$ CUDA_VISIBLE_DEVICES=0 python llama_inference.py ../alpaca-lora/hf_ckpt/  --wbits 4 --load alpace-7b-4bit-non-cleaned.pt --text  "Hello"
Loading model ...
Done.
 Hello Kub Akademutionsiy?')}{\rightarrow größ office \\ größ Roberts lé?'ulseulseame collect authorizationSide色սöd São affected throwingcur let authorizationמ bug affected dw collectmineною Chairulse instant Unionistrictscherरabsरclusowclar

The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.

/home/georgi/Documents/GitHub/alpaca-lora/venv/bin/python /home/georgi/Documents/GitHub/alpaca-lora/generate.py 

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
/home/georgi/Documents/GitHub/alpaca-lora/venv/lib/python3.10/site-packages/bitsandbytes/cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
CUDA SETUP: Required library version not found: libsbitsandbytes_cpu.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.
Traceback (most recent call last):
  File "/home/georgi/Documents/GitHub/alpaca-lora/generate.py", line 7, in <module>
    model = LlamaForCausalLM.from_pretrained(
  File "/home/georgi/Documents/GitHub/alpaca-lora/venv/lib/python3.10/site-packages/transformers/modeling_utils.py", line 2591, in from_pretrained
    raise ValueError(
ValueError: 
                        Some modules are dispatched on the CPU or the disk. Make sure you have enough GPU RAM to fit
                        the quantized model. If you want to dispatch the model on the CPU or the disk while keeping
                        these modules in 32-bit, you need to set `load_in_8bit_fp32_cpu_offload=True` and pass a custom
                        `device_map` to `from_pretrained`. Check
                        https://huggingface.co/docs/transformers/main/en/main_classes/quantization#offload-between-cpu-and-gpu
                        for more details.
                        

Process finished with exit code 1

Below is an instruction that describes a task, possibly paired with an input that provides further context. Write a response that appropriately completes the request.
<instruction>Add two numbers
<input>2, 3
<response>

The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /home/t-enshengshi/anaconda3/envs/alpaca-lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
CUDA SETUP: CUDA runtime path found: /usr/local/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.0
CUDA SETUP: Detected CUDA version 112
/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cuda112_nocublaslt.so...
Downloading tokenizer.model: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 500k/500k [00:00<00:00, 8.89MB/s]
Downloading (…)cial_tokens_map.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2.00/2.00 [00:00<00:00, 173B/s]
Downloading (…)okenizer_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 141/141 [00:00<00:00, 48.2kB/s]
Downloading (…)lve/main/config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 427/427 [00:00<00:00, 46.2kB/s]
Downloading (…)model.bin.index.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25.5k/25.5k [00:00<00:00, 333kB/s]
Downloading (…)l-00001-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:02<00:00, 169MB/s]
Downloading (…)l-00002-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 103MB/s]
Downloading (…)l-00003-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:07<00:00, 57.7MB/s]
Downloading (…)l-00004-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 118MB/s]
Downloading (…)l-00005-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 96.5MB/s]
Downloading (…)l-00006-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:07<00:00, 51.3MB/s]
Downloading (…)l-00007-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 104MB/s]
Downloading (…)l-00008-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 94.4MB/s]
Downloading (…)l-00009-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 97.8MB/s]
Downloading (…)l-00010-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 119MB/s]
Downloading (…)l-00011-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 122MB/s]
Downloading (…)l-00012-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 128MB/s]
Downloading (…)l-00013-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 92.4MB/s]
Downloading (…)l-00014-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 114MB/s]
Downloading (…)l-00015-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 109MB/s]
Downloading (…)l-00016-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 120MB/s]
Downloading (…)l-00017-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 120MB/s]
Downloading (…)l-00018-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 130MB/s]
Downloading (…)l-00019-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 84.2MB/s]
Downloading (…)l-00020-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 86.4MB/s]
Downloading (…)l-00021-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 99.9MB/s]
Downloading (…)l-00022-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 110MB/s]
Downloading (…)l-00023-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 105MB/s]
Downloading (…)l-00024-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 122MB/s]
Downloading (…)l-00025-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 117MB/s]
Downloading (…)l-00026-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 96.9MB/s]
Downloading (…)l-00027-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 129MB/s]
Downloading (…)l-00028-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 96.6MB/s]
Downloading (…)l-00029-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:02<00:00, 158MB/s]
Downloading (…)l-00030-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 101MB/s]
Downloading (…)l-00031-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:04<00:00, 100MB/s]
Downloading (…)l-00032-of-00033.bin: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 405M/405M [00:03<00:00, 108MB/s]
Downloading (…)l-00033-of-00033.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 524M/524M [00:05<00:00, 94.0MB/s]
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:08<00:00,  3.78it/s]
Some weights of the model checkpoint at decapoda-research/llama-7b-hf were not used when initializing LLaMAForCausalLM: ['model.layers.6.input_layernorm.weight', 'model.layers.11.self_attn.rotary_emb.inv_freq', 'model.layers.19.mlp.down_proj.weight', 'model.layers.19.self_attn.v_proj.weight', 'model.layers.28.self_attn.k_proj.weight', 'model.layers.10.mlp.down_proj.weight', 'model.layers.24.self_attn.rotary_emb.inv_freq', 'model.layers.10.mlp.up_proj.weight', 'model.layers.19.input_layernorm.weight', 'model.layers.28.post_attention_layernorm.weight', 'model.layers.22.self_attn.rotary_emb.inv_freq', 'model.layers.11.self_attn.k_proj.weight', 'model.layers.18.self_attn.q_proj.weight', 'model.layers.24.self_attn.q_proj.weight', 'model.layers.11.self_attn.o_proj.weight', 'model.layers.1.post_attention_layernorm.weight', 'model.layers.4.mlp.gate_proj.weight', 'model.layers.17.self_attn.o_proj.weight', 'model.layers.6.self_attn.o_proj.weight', 'model.layers.31.self_attn.rotary_emb.inv_freq', 'model.layers.5.post_attention_layernorm.weight', 'model.layers.30.mlp.gate_proj.weight', 'model.layers.19.self_attn.rotary_emb.inv_freq', 'model.layers.14.self_attn.k_proj.weight', 'model.layers.27.self_attn.o_proj.weight', 'model.layers.20.self_attn.o_proj.weight', 'model.layers.19.post_attention_layernorm.weight', 'model.layers.1.mlp.down_proj.weight', 'model.layers.3.post_attention_layernorm.weight', 'model.layers.16.self_attn.v_proj.weight', 'model.layers.8.self_attn.k_proj.weight', 'model.layers.9.self_attn.k_proj.weight', 'model.layers.21.mlp.down_proj.weight', 'model.layers.12.self_attn.o_proj.weight', 'model.layers.4.self_attn.v_proj.weight', 'model.layers.19.self_attn.k_proj.weight', 'model.layers.3.mlp.gate_proj.weight', 'model.layers.12.self_attn.q_proj.weight', 'model.layers.13.self_attn.q_proj.weight', 'model.layers.20.post_attention_layernorm.weight', 'model.layers.5.mlp.gate_proj.weight', 'model.layers.12.self_attn.rotary_emb.inv_freq', 'model.layers.4.mlp.up_proj.weight', 'model.layers.3.self_attn.rotary_emb.inv_freq', 'model.layers.23.self_attn.v_proj.weight', 'model.layers.0.mlp.gate_proj.weight', 'model.layers.11.post_attention_layernorm.weight', 'model.layers.18.mlp.up_proj.weight', 'model.layers.15.self_attn.rotary_emb.inv_freq', 'model.layers.7.mlp.down_proj.weight', 'model.layers.21.mlp.gate_proj.weight', 'model.layers.23.mlp.down_proj.weight', 'model.layers.31.input_layernorm.weight', 'model.layers.7.self_attn.k_proj.weight', 'model.layers.5.mlp.up_proj.weight', 'model.layers.5.self_attn.rotary_emb.inv_freq', 'model.layers.22.self_attn.k_proj.weight', 'model.layers.8.mlp.down_proj.weight', 'model.layers.10.self_attn.rotary_emb.inv_freq', 'model.layers.27.self_attn.rotary_emb.inv_freq', 'model.layers.31.self_attn.q_proj.weight', 'model.layers.22.self_attn.q_proj.weight', 'model.layers.15.self_attn.v_proj.weight', 'model.layers.24.self_attn.v_proj.weight', 'model.layers.19.mlp.gate_proj.weight', 'model.layers.25.mlp.down_proj.weight', 'model.layers.28.self_attn.rotary_emb.inv_freq', 'model.layers.6.mlp.down_proj.weight', 'model.layers.7.input_layernorm.weight', 'model.layers.14.mlp.down_proj.weight', 'model.layers.26.self_attn.q_proj.weight', 'model.layers.25.self_attn.v_proj.weight', 'model.layers.1.self_attn.k_proj.weight', 'model.layers.13.post_attention_layernorm.weight', 'model.layers.28.self_attn.o_proj.weight', 'model.layers.24.input_layernorm.weight', 'model.layers.0.input_layernorm.weight', 'model.layers.28.input_layernorm.weight', 'model.norm.weight', 'model.layers.11.self_attn.q_proj.weight', 'model.layers.12.mlp.down_proj.weight', 'model.layers.18.post_attention_layernorm.weight', 'model.layers.29.self_attn.v_proj.weight', 'model.layers.29.mlp.down_proj.weight', 'model.layers.26.self_attn.o_proj.weight', 'model.layers.2.input_layernorm.weight', 'model.layers.25.mlp.up_proj.weight', 'model.layers.17.mlp.down_proj.weight', 'model.layers.7.self_attn.v_proj.weight', 'model.layers.12.self_attn.v_proj.weight', 'model.layers.5.self_attn.k_proj.weight', 'model.layers.29.input_layernorm.weight', 'model.layers.1.self_attn.rotary_emb.inv_freq', 'model.layers.24.mlp.down_proj.weight', 'model.layers.27.mlp.down_proj.weight', 'model.layers.0.self_attn.q_proj.weight', 'model.layers.15.mlp.down_proj.weight', 'model.layers.7.mlp.gate_proj.weight', 'model.layers.2.self_attn.o_proj.weight', 'model.layers.22.post_attention_layernorm.weight', 'model.layers.29.self_attn.rotary_emb.inv_freq', 'model.layers.11.mlp.up_proj.weight', 'model.layers.31.mlp.up_proj.weight', 'model.layers.13.mlp.gate_proj.weight', 'model.layers.18.mlp.down_proj.weight', 'model.layers.21.self_attn.q_proj.weight', 'model.layers.21.mlp.up_proj.weight', 'model.layers.8.self_attn.v_proj.weight', 'model.layers.11.self_attn.v_proj.weight', 'model.layers.15.mlp.up_proj.weight', 'model.layers.11.mlp.gate_proj.weight', 'model.layers.26.self_attn.rotary_emb.inv_freq', 'model.layers.17.mlp.gate_proj.weight', 'model.layers.28.self_attn.q_proj.weight', 'model.layers.6.mlp.up_proj.weight', 'model.layers.26.post_attention_layernorm.weight', 'model.layers.2.mlp.up_proj.weight', 'model.layers.2.self_attn.v_proj.weight', 'model.layers.24.post_attention_layernorm.weight', 'model.layers.18.self_attn.k_proj.weight', 'model.layers.7.mlp.up_proj.weight', 'model.layers.26.self_attn.v_proj.weight', 'model.layers.22.self_attn.v_proj.weight', 'model.layers.13.self_attn.v_proj.weight', 'model.layers.1.self_attn.o_proj.weight', 'model.layers.19.self_attn.o_proj.weight', 'model.layers.15.self_attn.q_proj.weight', 'model.layers.9.self_attn.o_proj.weight', 'model.layers.11.mlp.down_proj.weight', 'model.layers.23.self_attn.q_proj.weight', 'model.layers.30.input_layernorm.weight', 'model.layers.8.input_layernorm.weight', 'model.layers.26.mlp.up_proj.weight', 'model.layers.4.mlp.down_proj.weight', 'model.layers.22.mlp.up_proj.weight', 'model.layers.25.post_attention_layernorm.weight', 'model.layers.27.mlp.up_proj.weight', 'model.layers.23.self_attn.rotary_emb.inv_freq', 'model.layers.3.self_attn.v_proj.weight', 'model.layers.20.input_layernorm.weight', 'model.layers.29.self_attn.k_proj.weight', 'model.layers.26.mlp.gate_proj.weight', 'model.layers.9.self_attn.v_proj.weight', 'model.layers.3.self_attn.k_proj.weight', 'model.layers.23.self_attn.o_proj.weight', 'model.layers.17.self_attn.q_proj.weight', 'model.layers.26.self_attn.k_proj.weight', 'model.layers.31.self_attn.v_proj.weight', 'model.layers.1.mlp.up_proj.weight', 'model.layers.10.input_layernorm.weight', 'model.layers.19.self_attn.q_proj.weight', 'model.layers.30.mlp.down_proj.weight', 'model.layers.22.mlp.gate_proj.weight', 'model.layers.3.input_layernorm.weight', 'model.layers.25.self_attn.q_proj.weight', 'model.layers.17.self_attn.v_proj.weight', 'model.layers.22.input_layernorm.weight', 'model.layers.0.self_attn.rotary_emb.inv_freq', 'model.layers.24.mlp.gate_proj.weight', 'model.layers.5.input_layernorm.weight', 'model.layers.16.post_attention_layernorm.weight', 'model.layers.15.self_attn.k_proj.weight', 'model.layers.18.input_layernorm.weight', 'model.layers.12.mlp.gate_proj.weight', 'model.layers.16.mlp.down_proj.weight', 'model.layers.14.mlp.up_proj.weight', 'model.layers.21.self_attn.k_proj.weight', 'model.layers.16.self_attn.k_proj.weight', 'model.layers.10.self_attn.q_proj.weight', 'model.layers.9.mlp.down_proj.weight', 'model.layers.16.input_layernorm.weight', 'model.layers.31.post_attention_layernorm.weight', 'model.layers.3.mlp.up_proj.weight', 'model.layers.29.post_attention_layernorm.weight', 'model.layers.13.mlp.down_proj.weight', 'model.layers.4.self_attn.rotary_emb.inv_freq', 'model.layers.27.mlp.gate_proj.weight', 'model.layers.6.mlp.gate_proj.weight', 'model.layers.8.mlp.gate_proj.weight', 'model.layers.25.self_attn.k_proj.weight', 'model.layers.14.self_attn.rotary_emb.inv_freq', 'model.layers.26.input_layernorm.weight', 'model.layers.21.post_attention_layernorm.weight', 'model.layers.6.post_attention_layernorm.weight', 'model.layers.15.self_attn.o_proj.weight', 'model.layers.23.mlp.gate_proj.weight', 'model.layers.2.post_attention_layernorm.weight', 'model.layers.4.post_attention_layernorm.weight', 'model.layers.9.input_layernorm.weight', 'model.layers.16.self_attn.rotary_emb.inv_freq', 'model.layers.0.self_attn.k_proj.weight', 'model.layers.21.self_attn.rotary_emb.inv_freq', 'model.layers.27.self_attn.k_proj.weight', 'model.layers.6.self_attn.rotary_emb.inv_freq', 'model.layers.28.mlp.down_proj.weight', 'model.layers.12.self_attn.k_proj.weight', 'model.layers.13.input_layernorm.weight', 'model.layers.10.mlp.gate_proj.weight', 'model.layers.31.self_attn.o_proj.weight', 'model.layers.27.self_attn.q_proj.weight', 'model.layers.18.self_attn.rotary_emb.inv_freq', 'model.layers.0.self_attn.o_proj.weight', 'model.layers.0.post_attention_layernorm.weight', 'model.layers.12.mlp.up_proj.weight', 'model.layers.27.input_layernorm.weight', 'model.layers.24.mlp.up_proj.weight', 'model.layers.13.mlp.up_proj.weight', 'model.layers.23.mlp.up_proj.weight', 'model.layers.9.self_attn.rotary_emb.inv_freq', 'model.layers.25.input_layernorm.weight', 'model.layers.25.self_attn.o_proj.weight', 'model.layers.29.self_attn.o_proj.weight', 'model.layers.30.self_attn.q_proj.weight', 'model.layers.13.self_attn.rotary_emb.inv_freq', 'model.layers.5.self_attn.v_proj.weight', 'model.layers.12.input_layernorm.weight', 'model.layers.31.mlp.gate_proj.weight', 'model.layers.6.self_attn.q_proj.weight', 'model.layers.7.self_attn.rotary_emb.inv_freq', 'model.layers.30.self_attn.v_proj.weight', 'model.layers.31.mlp.down_proj.weight', 'model.layers.10.self_attn.v_proj.weight', 'model.layers.18.self_attn.o_proj.weight', 'model.layers.17.mlp.up_proj.weight', 'model.layers.30.mlp.up_proj.weight', 'model.layers.7.self_attn.o_proj.weight', 'model.layers.18.mlp.gate_proj.weight', 'model.layers.18.self_attn.v_proj.weight', 'model.layers.22.mlp.down_proj.weight', 'model.layers.28.mlp.up_proj.weight', 'model.layers.6.self_attn.k_proj.weight', 'model.layers.14.post_attention_layernorm.weight', 'model.layers.28.mlp.gate_proj.weight', 'model.layers.23.self_attn.k_proj.weight', 'model.embed_tokens.weight', 'model.layers.7.self_attn.q_proj.weight', 'model.layers.29.mlp.gate_proj.weight', 'model.layers.25.self_attn.rotary_emb.inv_freq', 'model.layers.21.self_attn.v_proj.weight', 'model.layers.8.mlp.up_proj.weight', 'model.layers.14.self_attn.o_proj.weight', 'model.layers.20.mlp.gate_proj.weight', 'model.layers.16.mlp.gate_proj.weight', 'model.layers.16.self_attn.q_proj.weight', 'model.layers.30.self_attn.rotary_emb.inv_freq', 'model.layers.16.mlp.up_proj.weight', 'model.layers.20.self_attn.k_proj.weight', 'model.layers.11.input_layernorm.weight', 'model.layers.26.mlp.down_proj.weight', 'model.layers.9.post_attention_layernorm.weight', 'model.layers.1.input_layernorm.weight', 'model.layers.21.self_attn.o_proj.weight', 'model.layers.17.post_attention_layernorm.weight', 'model.layers.20.mlp.up_proj.weight', 'model.layers.2.self_attn.q_proj.weight', 'model.layers.15.mlp.gate_proj.weight', 'model.layers.5.self_attn.o_proj.weight', 'model.layers.1.mlp.gate_proj.weight', 'model.layers.4.self_attn.q_proj.weight', 'model.layers.2.mlp.gate_proj.weight', 'model.layers.20.self_attn.v_proj.weight', 'model.layers.7.post_attention_layernorm.weight', 'model.layers.10.self_attn.k_proj.weight', 'model.layers.14.input_layernorm.weight', 'model.layers.10.post_attention_layernorm.weight', 'model.layers.27.self_attn.v_proj.weight', 'model.layers.3.self_attn.o_proj.weight', 'model.layers.5.mlp.down_proj.weight', 'model.layers.29.self_attn.q_proj.weight', 'model.layers.8.self_attn.q_proj.weight', 'model.layers.28.self_attn.v_proj.weight', 'model.layers.0.self_attn.v_proj.weight', 'model.layers.4.self_attn.k_proj.weight', 'model.layers.14.self_attn.q_proj.weight', 'model.layers.23.post_attention_layernorm.weight', 'model.layers.27.post_attention_layernorm.weight', 'model.layers.29.mlp.up_proj.weight', 'model.layers.19.mlp.up_proj.weight', 'model.layers.20.mlp.down_proj.weight', 'model.layers.0.mlp.up_proj.weight', 'model.layers.30.self_attn.o_proj.weight', 'model.layers.25.mlp.gate_proj.weight', 'model.layers.13.self_attn.o_proj.weight', 'model.layers.4.input_layernorm.weight', 'model.layers.0.mlp.down_proj.weight', 'model.layers.20.self_attn.rotary_emb.inv_freq', 'model.layers.30.post_attention_layernorm.weight', 'model.layers.20.self_attn.q_proj.weight', 'model.layers.2.self_attn.rotary_emb.inv_freq', 'model.layers.1.self_attn.q_proj.weight', 'model.layers.17.self_attn.k_proj.weight', 'model.layers.24.self_attn.k_proj.weight', 'model.layers.4.self_attn.o_proj.weight', 'model.layers.9.self_attn.q_proj.weight', 'model.layers.9.mlp.gate_proj.weight', 'model.layers.30.self_attn.k_proj.weight', 'model.layers.14.self_attn.v_proj.weight', 'model.layers.22.self_attn.o_proj.weight', 'model.layers.31.self_attn.k_proj.weight', 'model.layers.24.self_attn.o_proj.weight', 'model.layers.2.mlp.down_proj.weight', 'model.layers.2.self_attn.k_proj.weight', 'model.layers.14.mlp.gate_proj.weight', 'model.layers.3.self_attn.q_proj.weight', 'model.layers.8.self_attn.rotary_emb.inv_freq', 'model.layers.10.self_attn.o_proj.weight', 'model.layers.8.self_attn.o_proj.weight', 'model.layers.3.mlp.down_proj.weight', 'model.layers.15.post_attention_layernorm.weight', 'model.layers.12.post_attention_layernorm.weight', 'model.layers.17.self_attn.rotary_emb.inv_freq', 'model.layers.17.input_layernorm.weight', 'model.layers.6.self_attn.v_proj.weight', 'model.layers.15.input_layernorm.weight', 'model.layers.9.mlp.up_proj.weight', 'model.layers.21.input_layernorm.weight', 'model.layers.1.self_attn.v_proj.weight', 'model.layers.8.post_attention_layernorm.weight', 'model.layers.23.input_layernorm.weight', 'model.layers.5.self_attn.q_proj.weight', 'model.layers.16.self_attn.o_proj.weight', 'model.layers.13.self_attn.k_proj.weight']
- This IS expected if you are initializing LLaMAForCausalLM from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing LLaMAForCausalLM from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
Some weights of LLaMAForCausalLM were not initialized from the model checkpoint at decapoda-research/llama-7b-hf and are newly initialized: ['model.decoder.layers.25.feed_forward.w3.weight', 'model.decoder.layers.25.self_attn.k_proj.weight', 'model.decoder.layers.22.ffn_norm.weight', 'model.decoder.layers.23.self_attn.o_proj.weight', 'model.decoder.layers.22.self_attn.o_proj.weight', 'model.decoder.layers.26.feed_forward.w2.weight', 'model.decoder.layers.6.self_attn.o_proj.weight', 'model.decoder.layers.14.self_attn.q_proj.weight', 'model.decoder.layers.17.attention_norm.weight', 'model.decoder.layers.19.self_attn.o_proj.weight', 'model.decoder.layers.15.feed_forward.w1.weight', 'model.decoder.layers.21.feed_forward.w2.weight', 'model.decoder.layers.10.self_attn.o_proj.weight', 'model.decoder.layers.24.ffn_norm.weight', 'model.decoder.layers.11.feed_forward.w2.weight', 'model.decoder.layers.15.self_attn.k_proj.weight', 'model.decoder.layers.13.attention_norm.weight', 'model.decoder.layers.6.attention_norm.weight', 'model.decoder.layers.7.attention_norm.weight', 'model.decoder.layers.8.feed_forward.w2.weight', 'model.decoder.layers.18.self_attn.q_proj.weight', 'model.decoder.layers.26.feed_forward.w3.weight', 'model.decoder.layers.15.self_attn.o_proj.weight', 'model.decoder.layers.28.attention_norm.weight', 'model.decoder.layers.31.self_attn.o_proj.weight', 'model.decoder.layers.16.self_attn.o_proj.weight', 'model.decoder.layers.17.self_attn.k_proj.weight', 'model.decoder.layers.13.self_attn.q_proj.weight', 'model.decoder.layers.21.self_attn.o_proj.weight', 'model.decoder.layers.28.self_attn.v_proj.weight', 'model.decoder.layers.30.self_attn.o_proj.weight', 'model.decoder.layers.1.self_attn.o_proj.weight', 'model.decoder.layers.15.self_attn.v_proj.weight', 'model.decoder.layers.1.feed_forward.w1.weight', 'model.decoder.layers.1.feed_forward.w3.weight', 'model.decoder.layers.8.self_attn.v_proj.weight', 'model.decoder.layers.21.self_attn.q_proj.weight', 'model.decoder.layers.3.self_attn.v_proj.weight', 'model.decoder.layers.18.ffn_norm.weight', 'model.decoder.layers.22.feed_forward.w3.weight', 'model.decoder.layers.27.ffn_norm.weight', 'model.decoder.layers.8.ffn_norm.weight', 'model.decoder.layers.8.self_attn.k_proj.weight', 'model.decoder.layers.24.feed_forward.w3.weight', 'model.decoder.layers.14.feed_forward.w3.weight', 'model.decoder.layers.16.attention_norm.weight', 'model.decoder.layers.5.feed_forward.w3.weight', 'model.decoder.layers.11.feed_forward.w3.weight', 'model.decoder.layers.4.attention_norm.weight', 'model.decoder.layers.21.ffn_norm.weight', 'model.decoder.layers.28.self_attn.o_proj.weight', 'model.decoder.layers.30.self_attn.k_proj.weight', 'model.decoder.layers.14.feed_forward.w1.weight', 'model.decoder.layers.16.feed_forward.w2.weight', 'model.decoder.layers.24.feed_forward.w2.weight', 'model.decoder.layers.6.self_attn.k_proj.weight', 'model.decoder.layers.20.attention_norm.weight', 'model.decoder.layers.15.attention_norm.weight', 'model.decoder.layers.3.self_attn.q_proj.weight', 'model.decoder.layers.17.self_attn.o_proj.weight', 'model.decoder.layers.25.feed_forward.w1.weight', 'model.decoder.layers.1.feed_forward.w2.weight', 'model.decoder.layers.19.attention_norm.weight', 'model.decoder.layers.13.feed_forward.w1.weight', 'model.decoder.layers.1.self_attn.k_proj.weight', 'model.decoder.layers.20.feed_forward.w2.weight', 'model.decoder.layers.17.ffn_norm.weight', 'model.decoder.layers.12.attention_norm.weight', 'model.decoder.layers.23.ffn_norm.weight', 'model.decoder.layers.14.self_attn.k_proj.weight', 'model.decoder.layers.26.feed_forward.w1.weight', 'model.decoder.layers.6.feed_forward.w1.weight', 'model.decoder.layers.12.feed_forward.w3.weight', 'model.decoder.layers.0.feed_forward.w3.weight', 'model.decoder.layers.22.feed_forward.w1.weight', 'model.decoder.layers.29.feed_forward.w1.weight', 'model.decoder.layers.19.self_attn.k_proj.weight', 'model.decoder.layers.11.self_attn.v_proj.weight', 'model.decoder.layers.13.feed_forward.w3.weight', 'model.decoder.layers.4.ffn_norm.weight', 'model.decoder.layers.9.attention_norm.weight', 'model.decoder.layers.30.feed_forward.w3.weight', 'model.decoder.layers.17.feed_forward.w1.weight', 'model.decoder.layers.18.self_attn.v_proj.weight', 'model.decoder.layers.29.ffn_norm.weight', 'model.decoder.layers.17.self_attn.q_proj.weight', 'model.decoder.layers.23.feed_forward.w2.weight', 'model.decoder.layers.31.feed_forward.w2.weight', 'model.decoder.layers.11.attention_norm.weight', 'model.decoder.layers.5.feed_forward.w2.weight', 'model.decoder.layers.3.feed_forward.w1.weight', 'model.decoder.layers.29.self_attn.q_proj.weight', 'model.decoder.layers.15.feed_forward.w2.weight', 'model.decoder.layers.2.feed_forward.w1.weight', 'model.decoder.layers.27.feed_forward.w1.weight', 'model.decoder.layers.30.self_attn.q_proj.weight', 'model.decoder.layers.0.attention_norm.weight', 'model.decoder.layers.23.feed_forward.w3.weight', 'model.decoder.layers.5.self_attn.o_proj.weight', 'model.decoder.layers.14.self_attn.v_proj.weight', 'model.decoder.layers.4.feed_forward.w1.weight', 'model.decoder.layers.23.feed_forward.w1.weight', 'model.decoder.layers.29.self_attn.k_proj.weight', 'model.decoder.layers.2.self_attn.v_proj.weight', 'model.decoder.layers.3.ffn_norm.weight', 'model.decoder.layers.11.self_attn.k_proj.weight', 'model.decoder.layers.18.feed_forward.w3.weight', 'model.decoder.layers.27.feed_forward.w2.weight', 'model.decoder.layers.28.self_attn.k_proj.weight', 'model.decoder.layers.31.attention_norm.weight', 'model.decoder.layers.8.feed_forward.w3.weight', 'model.decoder.layers.9.feed_forward.w2.weight', 'model.decoder.layers.10.self_attn.v_proj.weight', 'model.decoder.layers.2.attention_norm.weight', 'model.decoder.layers.14.feed_forward.w2.weight', 'model.decoder.layers.19.feed_forward.w2.weight', 'model.decoder.layers.21.attention_norm.weight', 'model.decoder.layers.24.attention_norm.weight', 'model.decoder.layers.26.self_attn.k_proj.weight', 'model.decoder.layers.26.self_attn.v_proj.weight', 'model.decoder.layers.9.feed_forward.w1.weight', 'model.decoder.layers.31.feed_forward.w1.weight', 'model.decoder.layers.13.self_attn.k_proj.weight', 'model.decoder.layers.6.ffn_norm.weight', 'model.decoder.layers.8.attention_norm.weight', 'model.decoder.layers.19.feed_forward.w3.weight', 'model.decoder.layers.27.feed_forward.w3.weight', 'model.decoder.layers.18.self_attn.o_proj.weight', 'model.decoder.layers.27.self_attn.q_proj.weight', 'model.decoder.layers.8.self_attn.o_proj.weight', 'model.decoder.layers.27.self_attn.v_proj.weight', 'model.decoder.layers.9.self_attn.q_proj.weight', 'model.decoder.layers.6.self_attn.v_proj.weight', 'model.decoder.layers.0.ffn_norm.weight', 'model.decoder.layers.19.self_attn.v_proj.weight', 'model.decoder.layers.22.feed_forward.w2.weight', 'model.decoder.layers.29.self_attn.o_proj.weight', 'model.decoder.layers.1.self_attn.v_proj.weight', 'model.decoder.layers.11.ffn_norm.weight', 'model.decoder.layers.4.self_attn.q_proj.weight', 'model.decoder.layers.10.feed_forward.w3.weight', 'model.decoder.layers.18.self_attn.k_proj.weight', 'model.decoder.layers.16.self_attn.v_proj.weight', 'model.decoder.layers.7.feed_forward.w3.weight', 'model.decoder.layers.23.self_attn.v_proj.weight', 'model.decoder.layers.5.ffn_norm.weight', 'model.decoder.layers.9.feed_forward.w3.weight', 'model.decoder.layers.20.self_attn.k_proj.weight', 'model.decoder.norm.weight', 'model.decoder.layers.3.attention_norm.weight', 'model.decoder.layers.10.ffn_norm.weight', 'model.decoder.layers.17.feed_forward.w2.weight', 'model.decoder.layers.3.self_attn.k_proj.weight', 'model.decoder.layers.5.attention_norm.weight', 'model.decoder.layers.20.ffn_norm.weight', 'model.decoder.layers.4.feed_forward.w2.weight', 'model.decoder.layers.17.feed_forward.w3.weight', 'model.decoder.layers.20.self_attn.o_proj.weight', 'model.decoder.layers.13.self_attn.v_proj.weight', 'model.decoder.layers.22.self_attn.k_proj.weight', 'model.decoder.layers.30.feed_forward.w1.weight', 'model.decoder.layers.4.feed_forward.w3.weight', 'model.decoder.layers.24.feed_forward.w1.weight', 'model.decoder.layers.30.attention_norm.weight', 'model.decoder.layers.7.self_attn.k_proj.weight', 'model.decoder.layers.5.self_attn.v_proj.weight', 'model.decoder.layers.1.attention_norm.weight', 'model.decoder.layers.7.ffn_norm.weight', 'model.decoder.layers.21.self_attn.k_proj.weight', 'model.decoder.layers.31.ffn_norm.weight', 'model.decoder.layers.3.feed_forward.w2.weight', 'model.decoder.layers.7.self_attn.o_proj.weight', 'model.decoder.layers.21.self_attn.v_proj.weight', 'model.decoder.layers.0.self_attn.k_proj.weight', 'model.decoder.layers.19.feed_forward.w1.weight', 'model.decoder.layers.2.self_attn.k_proj.weight', 'model.decoder.layers.24.self_attn.o_proj.weight', 'model.decoder.layers.10.feed_forward.w2.weight', 'model.decoder.layers.9.self_attn.v_proj.weight', 'model.decoder.layers.10.attention_norm.weight', 'model.decoder.layers.23.self_attn.q_proj.weight', 'model.decoder.layers.13.self_attn.o_proj.weight', 'model.decoder.layers.16.feed_forward.w3.weight', 'model.decoder.layers.1.ffn_norm.weight', 'model.decoder.layers.20.feed_forward.w1.weight', 'model.decoder.layers.15.self_attn.q_proj.weight', 'model.decoder.layers.24.self_attn.q_proj.weight', 'model.decoder.layers.2.ffn_norm.weight', 'model.decoder.layers.15.feed_forward.w3.weight', 'model.decoder.layers.29.attention_norm.weight', 'model.decoder.layers.4.self_attn.k_proj.weight', 'model.decoder.layers.10.feed_forward.w1.weight', 'model.decoder.layers.15.ffn_norm.weight', 'model.decoder.layers.12.self_attn.o_proj.weight', 'model.decoder.layers.2.self_attn.o_proj.weight', 'model.decoder.layers.23.self_attn.k_proj.weight', 'model.decoder.layers.25.ffn_norm.weight', 'model.decoder.layers.12.feed_forward.w1.weight', 'model.decoder.layers.28.feed_forward.w3.weight', 'model.decoder.layers.16.self_attn.k_proj.weight', 'model.decoder.layers.28.ffn_norm.weight', 'model.decoder.layers.26.self_attn.q_proj.weight', 'model.decoder.layers.13.ffn_norm.weight', 'model.decoder.layers.20.self_attn.q_proj.weight', 'model.decoder.layers.20.self_attn.v_proj.weight', 'model.decoder.layers.13.feed_forward.w2.weight', 'model.decoder.layers.26.self_attn.o_proj.weight', 'model.decoder.layers.9.ffn_norm.weight', 'model.decoder.layers.24.self_attn.v_proj.weight', 'model.decoder.layers.25.self_attn.o_proj.weight', 'model.decoder.layers.6.self_attn.q_proj.weight', 'model.decoder.layers.25.self_attn.v_proj.weight', 'model.decoder.layers.28.feed_forward.w2.weight', 'model.decoder.layers.22.attention_norm.weight', 'model.decoder.layers.10.self_attn.q_proj.weight', 'model.decoder.layers.27.self_attn.o_proj.weight', 'model.decoder.embed_tokens.weight', 'model.decoder.layers.4.self_attn.o_proj.weight', 'model.decoder.layers.14.ffn_norm.weight', 'model.decoder.layers.25.feed_forward.w2.weight', 'model.decoder.layers.21.feed_forward.w1.weight', 'model.decoder.layers.7.feed_forward.w2.weight', 'model.decoder.layers.0.feed_forward.w1.weight', 'model.decoder.layers.25.attention_norm.weight', 'model.decoder.layers.25.self_attn.q_proj.weight', 'model.decoder.layers.27.attention_norm.weight', 'model.decoder.layers.7.feed_forward.w1.weight', 'model.decoder.layers.29.self_attn.v_proj.weight', 'model.decoder.layers.2.feed_forward.w3.weight', 'model.decoder.layers.17.self_attn.v_proj.weight', 'model.decoder.layers.12.feed_forward.w2.weight', 'model.decoder.layers.5.feed_forward.w1.weight', 'model.decoder.layers.3.self_attn.o_proj.weight', 'model.decoder.layers.6.feed_forward.w2.weight', 'model.decoder.layers.11.self_attn.o_proj.weight', 'model.decoder.layers.19.ffn_norm.weight', 'model.decoder.layers.9.self_attn.o_proj.weight', 'model.decoder.layers.18.feed_forward.w1.weight', 'model.decoder.layers.0.feed_forward.w2.weight', 'model.decoder.layers.26.ffn_norm.weight', 'model.decoder.layers.30.self_attn.v_proj.weight', 'model.decoder.layers.27.self_attn.k_proj.weight', 'model.decoder.layers.30.feed_forward.w2.weight', 'model.decoder.layers.31.self_attn.k_proj.weight', 'model.decoder.layers.19.self_attn.q_proj.weight', 'model.decoder.layers.30.ffn_norm.weight', 'model.decoder.layers.2.feed_forward.w2.weight', 'model.decoder.layers.8.self_attn.q_proj.weight', 'model.decoder.layers.22.self_attn.q_proj.weight', 'model.decoder.layers.29.feed_forward.w3.weight', 'model.decoder.layers.0.self_attn.o_proj.weight', 'model.decoder.layers.5.self_attn.q_proj.weight', 'model.decoder.layers.11.feed_forward.w1.weight', 'model.decoder.layers.12.self_attn.k_proj.weight', 'model.decoder.layers.12.ffn_norm.weight', 'model.decoder.layers.28.self_attn.q_proj.weight', 'model.decoder.layers.14.self_attn.o_proj.weight', 'model.decoder.layers.29.feed_forward.w2.weight', 'model.decoder.layers.11.self_attn.q_proj.weight', 'model.decoder.layers.3.feed_forward.w3.weight', 'model.decoder.layers.0.self_attn.v_proj.weight', 'model.decoder.layers.7.self_attn.v_proj.weight', 'model.decoder.layers.12.self_attn.q_proj.weight', 'model.decoder.layers.21.feed_forward.w3.weight', 'model.decoder.layers.23.attention_norm.weight', 'model.decoder.layers.16.ffn_norm.weight', 'model.decoder.layers.9.self_attn.k_proj.weight', 'model.decoder.layers.10.self_attn.k_proj.weight', 'model.decoder.layers.31.feed_forward.w3.weight', 'model.decoder.layers.0.self_attn.q_proj.weight', 'model.decoder.layers.2.self_attn.q_proj.weight', 'model.decoder.layers.18.feed_forward.w2.weight', 'model.decoder.layers.18.attention_norm.weight', 'model.decoder.layers.1.self_attn.q_proj.weight', 'model.decoder.layers.4.self_attn.v_proj.weight', 'model.decoder.layers.22.self_attn.v_proj.weight', 'model.decoder.layers.24.self_attn.k_proj.weight', 'model.decoder.layers.6.feed_forward.w3.weight', 'model.decoder.layers.31.self_attn.v_proj.weight', 'model.decoder.layers.16.feed_forward.w1.weight', 'model.decoder.layers.16.self_attn.q_proj.weight', 'model.decoder.layers.5.self_attn.k_proj.weight', 'model.decoder.layers.20.feed_forward.w3.weight', 'model.decoder.layers.12.self_attn.v_proj.weight', 'model.decoder.layers.8.feed_forward.w1.weight', 'model.decoder.layers.28.feed_forward.w1.weight', 'model.decoder.layers.31.self_attn.q_proj.weight', 'model.decoder.layers.14.attention_norm.weight', 'model.decoder.layers.7.self_attn.q_proj.weight', 'model.decoder.layers.26.attention_norm.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
Downloading (…)neration_config.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 124/124 [00:00<00:00, 24.8kB/s]
Downloading (…)/adapter_config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 370/370 [00:00<00:00, 163kB/s]
Downloading adapter_model.bin: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16.8M/16.8M [00:00<00:00, 89.7MB/s]
Instruction: Tell me about alpacas.
Traceback (most recent call last):
  File "/home/t-enshengshi/workspace/alpaca-lora/generate.py", line 77, in <module>
    print("Response:", evaluate(instruction))
  File "/home/t-enshengshi/workspace/alpaca-lora/generate.py", line 51, in evaluate
    generation_output = model.generate(
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/peft/peft_model.py", line 581, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 1490, in generate
    return self.beam_search(
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/generation/utils.py", line 2749, in beam_search
    outputs = self(
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 852, in forward
    outputs = self.model.decoder(
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 624, in forward
    layer_outputs = decoder_layer(
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 305, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/models/llama/modeling_llama.py", line 165, in forward
    query_states = self.q_proj(hidden_states).view(bsz, tgt_len, self.num_heads, self.head_dim)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/peft/tuners/lora.py", line 522, in forward
    result = super().forward(x)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/nn/modules.py", line 242, in forward
    out = bnb.matmul(x, self.weight, bias=self.bias, state=self.state)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/autograd/_functions.py", line 488, in matmul
    return MatMul8bitLt.apply(A, B, out, bias, state)
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/home/t-enshengshi/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/autograd/_functions.py", line 390, in forward
    output = torch.nn.functional.linear(A_wo_outliers, state.CB.to(A.dtype))
AttributeError: 'NoneType' object has no attribute 'to'

generation_output = model.generate(
            input_ids=input_ids,
            generation_config=generation_config,
            return_dict_in_generate=True,
            output_scores=True,
            max_new_tokens=2048,
        )

/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/generation/utils.py:1374: UserWarning: You are calling .generate() with the `input_ids` being on a device type different than your m
(conda_alpaca) jovyan@ranking-0:~/alpaca-lora$ python3 test.py 

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
CUDA SETUP: CUDA runtime path found: /opt/conda/envs/conda_alpaca/lib/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 6.1
CUDA SETUP: Detected CUDA version 118
/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU!
  warn(msg)
CUDA SETUP: Loading binary /opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/bitsandbytes/libbitsandbytes_cuda118_nocublaslt.so...
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 33/33 [00:14<00:00,  2.22it/s]
Instruction: Tell me about alpacas.
Traceback (most recent call last):
  File "test.py", line 90, in <module>
    print("Response:", evaluate(instruction))
  File "test.py", line 47, in evaluate
    generation_output = model.generate(
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/peft/peft_model.py", line 581, in generate
    outputs = self.base_model.generate(**kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/generation/utils.py", line 1490, in generate
    return self.beam_search(
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/generation/utils.py", line 2749, in beam_search
    outputs = self(
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 772, in forward
    outputs = self.model(
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 621, in forward
    layer_outputs = decoder_layer(
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 316, in forward
    hidden_states, self_attn_weights, present_key_value = self.self_attn(
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/transformers/models/llama/modeling_llama.py", line 216, in forward
    query_states = self.q_proj(hidden_states).view(bsz, q_len, self.num_heads, self.head_dim).transpose(1, 2)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
    return forward_call(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/accelerate/hooks.py", line 165, in new_forward
    output = old_forward(*args, **kwargs)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/peft/tuners/lora.py", line 522, in forward
    result = super().forward(x)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/bitsandbytes/nn/modules.py", line 242, in forward
    out = bnb.matmul(x, self.weight, bias=self.bias, state=self.state)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 488, in matmul
    return MatMul8bitLt.apply(A, B, out, bias, state)
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/torch/autograd/function.py", line 506, in apply
    return super().apply(*args, **kwargs)  # type: ignore[misc]
  File "/opt/conda/envs/conda_alpaca/lib/python3.8/site-packages/bitsandbytes/autograd/_functions.py", line 360, in forward
    outliers = state.CB[:, state.idx.long()].clone()
TypeError: 'NoneType' object is not subscriptable

Something went wrong
Unexpected token '<', "<html> <h"... is not valid JSON

Write a Python program that prints the first 10 Fibonacci numbers.
write python code that renames all files starting with "mleml" in the current folder to "kek". 

===================================BUG REPORT===================================
Welcome to bitsandbytes. For bug reports, please submit your error trace to: https://github.com/TimDettmers/bitsandbytes/issues
================================================================================
/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: /root/anaconda3/envs/alpaca-lora did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: The following directories listed in your path were found to be non-existent: {PosixPath('/usr/local/cuda/lib64')}
  warn(msg)
/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: :/usr/local/cuda/lib64/ did not contain libcudart.so as expected! Searching further paths...
  warn(msg)
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching /usr/local/cuda/lib64...
ERROR: python: undefined symbol: cudaRuntimeGetVersion
CUDA SETUP: libcudart.so path is None
CUDA SETUP: Is seems that your cuda installation is not in your path. See https://github.com/TimDettmers/bitsandbytes/issues/85 for more information.
CUDA SETUP: CUDA version lower than 11 are currently not supported for LLM.int8(). You will be only to use 8-bit optimizers and quantization routines!!
/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: No libcudart.so found! Install CUDA or the cudatoolkit package (anaconda)!
  warn(msg)
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 00
CUDA SETUP: Loading binary /root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cpu.so...
/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/cextension.py:31: UserWarning: The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers and GPU quantization are unavailable.
  warn("The installed version of bitsandbytes was compiled without GPU support. "
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization. 
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'. 
The class this function is called from is 'LlamaTokenizer'.
Loading checkpoint shards:   0%|                                                                                                                   | 0/33 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/root/projects/alpaca-lora/generate.py", line 13, in <module>
    model = LlamaForCausalLM.from_pretrained(
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2646, in from_pretrained
    ) = cls._load_pretrained_model(
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/modeling_utils.py", line 2969, in _load_pretrained_model
    new_error_msgs, offload_index, state_dict_index = _load_state_dict_into_meta_model(
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/modeling_utils.py", line 676, in _load_state_dict_into_meta_model
    set_module_8bit_tensor_to_device(model, param_name, param_device, value=param)
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/transformers/utils/bitsandbytes.py", line 70, in set_module_8bit_tensor_to_device
    new_value = bnb.nn.Int8Params(new_value, requires_grad=False, has_fp16_weights=has_fp16_weights).to(device)
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/nn/modules.py", line 196, in to
    return self.cuda(device)
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/nn/modules.py", line 160, in cuda
    CB, CBt, SCB, SCBt, coo_tensorB = bnb.functional.double_quant(B)
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/functional.py", line 1616, in double_quant
    row_stats, col_stats, nnz_row_ptr = get_colrow_absmax(
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/functional.py", line 1505, in get_colrow_absmax
    lib.cget_col_row_stats(ptrA, ptrRowStats, ptrColStats, ptrNnzrows, ct.c_float(threshold), rows, cols)
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/ctypes/__init__.py", line 395, in __getattr__
    func = self.__getitem__(name)
  File "/root/anaconda3/envs/alpaca-lora/lib/python3.9/ctypes/__init__.py", line 400, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /root/anaconda3/envs/alpaca-lora/lib/python3.9/site-packages/bitsandbytes/libbitsandbytes_cpu.so: undefined symbol: cget_col_row_stats

git clone https://github.com/TimDettmers/bitsandbytes.git
cd bitsandbytes/
make cuda11x
CUDA_VERSION=112
python setup.py install

Fri Mar 17 13:53:10 2023       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.03   Driver Version: 470.161.03   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro RTX 4000     On   | 00000000:00:10.0 Off |                  N/A |
| 30%   34C    P8     6W / 125W |      1MiB /  7982MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Sun_Feb_14_21:12:58_PST_2021
Cuda compilation tools, release 11.2, V11.2.152
Build cuda_11.2.r11.2/compiler.29618528_0

which nvcc:
/usr/bin/nvcc
which nvidia-smi:
/usr/bin/nvidia-smi

model = LLaMAForCausalLM.from_pretrained(
    "decapoda-research/llama-7b-hf",
    load_in_8bit=True,
    device_map="auto",
)

    # packages in environment at /home/zhuji/miniconda3/envs/LiuJieTest:
    #
    # Name                    Version                   Build  Channel
    _libgcc_mutex             0.1                        main  
    _openmp_mutex             5.1                       1_gnu  
    accelerate                0.17.1                   pypi_0    pypi
    aiohttp                   3.8.4                    pypi_0    pypi
    aiosignal                 1.3.1                    pypi_0    pypi
    anyio                     3.6.2                    pypi_0    pypi
    argon2-cffi               21.3.0                   pypi_0    pypi
    argon2-cffi-bindings      21.2.0                   pypi_0    pypi
    arrow                     1.2.3                    pypi_0    pypi
    asgiref                   3.6.0                    pypi_0    pypi
    asttokens                 2.2.1                    pypi_0    pypi
    async-timeout             4.0.2                    pypi_0    pypi
    attrs                     22.2.0                   pypi_0    pypi
    backcall                  0.2.0                    pypi_0    pypi
    beautifulsoup4            4.11.2                   pypi_0    pypi
    bitsandbytes              0.37.1                   pypi_0    pypi
    bleach                    6.0.0                    pypi_0    pypi
    ca-certificates           2023.01.10           h06a4308_0  
    certifi                   2022.12.7        py39h06a4308_0  
    cffi                      1.15.1                   pypi_0    pypi
    charset-normalizer        3.1.0                    pypi_0    pypi
    cmake                     3.26.0                   pypi_0    pypi
    comm                      0.1.2                    pypi_0    pypi
    datasets                  2.10.1                   pypi_0    pypi
    debugpy                   1.6.6                    pypi_0    pypi
    decorator                 5.1.1                    pypi_0    pypi
    defusedxml                0.7.1                    pypi_0    pypi
    dill                      0.3.6                    pypi_0    pypi
    django                    4.1.7                    pypi_0    pypi
    executing                 1.2.0                    pypi_0    pypi
    fastjsonschema            2.16.3                   pypi_0    pypi
    filelock                  3.10.0                   pypi_0    pypi
    fqdn                      1.5.1                    pypi_0    pypi
    frozenlist                1.3.3                    pypi_0    pypi
    fsspec                    2023.3.0                 pypi_0    pypi
    huggingface-hub           0.13.2                   pypi_0    pypi
    idna                      3.4                      pypi_0    pypi
    importlib-metadata        6.0.0                    pypi_0    pypi
    ipykernel                 6.21.3                   pypi_0    pypi
    ipython                   8.11.0                   pypi_0    pypi
    ipython-genutils          0.2.0                    pypi_0    pypi
    ipywidgets                8.0.4                    pypi_0    pypi
    isoduration               20.11.0                  pypi_0    pypi
    jedi                      0.18.2                   pypi_0    pypi
    jinja2                    3.1.2                    pypi_0    pypi
    jsonpointer               2.3                      pypi_0    pypi
    jsonschema                4.17.3                   pypi_0    pypi
    jupyter                   1.0.0                    pypi_0    pypi
    jupyter-client            8.0.3                    pypi_0    pypi
    jupyter-console           6.6.3                    pypi_0    pypi
    jupyter-core              5.2.0                    pypi_0    pypi
    jupyter-events            0.6.3                    pypi_0    pypi
    jupyter-server            2.4.0                    pypi_0    pypi
    jupyter-server-terminals  0.4.4                    pypi_0    pypi
    jupyterlab-pygments       0.2.2                    pypi_0    pypi
    jupyterlab-widgets        3.0.5                    pypi_0    pypi
    ld_impl_linux-64          2.38                 h1181459_1  
    libffi                    3.4.2                h6a678d5_6  
    libgcc-ng                 11.2.0               h1234567_1  
    libgomp                   11.2.0               h1234567_1  
    libstdcxx-ng              11.2.0               h1234567_1  
    lit                       15.0.7                   pypi_0    pypi
    loralib                   0.1.1                    pypi_0    pypi
    markupsafe                2.1.2                    pypi_0    pypi
    matplotlib-inline         0.1.6                    pypi_0    pypi
    mistune                   2.0.5                    pypi_0    pypi
    mpmath                    1.3.0                    pypi_0    pypi
    multidict                 6.0.4                    pypi_0    pypi
    multiprocess              0.70.14                  pypi_0    pypi
    nbclassic                 0.5.3                    pypi_0    pypi
    nbclient                  0.7.2                    pypi_0    pypi
    nbconvert                 7.2.10                   pypi_0    pypi
    nbformat                  5.7.3                    pypi_0    pypi
    ncurses                   6.4                  h6a678d5_0  
    nest-asyncio              1.5.6                    pypi_0    pypi
    networkx                  3.0                      pypi_0    pypi
    notebook                  6.5.3                    pypi_0    pypi
    notebook-shim             0.2.2                    pypi_0    pypi
    numpy                     1.24.2                   pypi_0    pypi
    nvidia-cublas-cu11        11.10.3.66               pypi_0    pypi
    nvidia-cuda-cupti-cu11    11.7.101                 pypi_0    pypi
    nvidia-cuda-nvrtc-cu11    11.7.99                  pypi_0    pypi
    nvidia-cuda-runtime-cu11  11.7.99                  pypi_0    pypi
    nvidia-cudnn-cu11         8.5.0.96                 pypi_0    pypi
    nvidia-cufft-cu11         10.9.0.58                pypi_0    pypi
    nvidia-curand-cu11        10.2.10.91               pypi_0    pypi
    nvidia-cusolver-cu11      11.4.0.1                 pypi_0    pypi
    nvidia-cusparse-cu11      11.7.4.91                pypi_0    pypi
    nvidia-nccl-cu11          2.14.3                   pypi_0    pypi
    nvidia-nvtx-cu11          11.7.91                  pypi_0    pypi
    openssl                   1.1.1t               h7f8727e_0  
    packaging                 23.0                     pypi_0    pypi
    pandas                    1.5.3                    pypi_0    pypi
    pandocfilters             1.5.0                    pypi_0    pypi
    parso                     0.8.3                    pypi_0    pypi
    peft                      0.3.0.dev0               pypi_0    pypi
    pexpect                   4.8.0                    pypi_0    pypi
    pickleshare               0.7.5                    pypi_0    pypi
    pip                       23.0.1           py39h06a4308_0  
    platformdirs              3.1.1                    pypi_0    pypi
    prometheus-client         0.16.0                   pypi_0    pypi
    prompt-toolkit            3.0.38                   pypi_0    pypi
    psutil                    5.9.4                    pypi_0    pypi
    ptyprocess                0.7.0                    pypi_0    pypi
    pure-eval                 0.2.2                    pypi_0    pypi
    pyarrow                   11.0.0                   pypi_0    pypi
    pycparser                 2.21                     pypi_0    pypi
    pygments                  2.14.0                   pypi_0    pypi
    pyrsistent                0.19.3                   pypi_0    pypi
    python                    3.9.16               h7a1cb2a_2  
    python-dateutil           2.8.2                    pypi_0    pypi
    python-json-logger        2.0.7                    pypi_0    pypi
    pytz                      2022.7.1                 pypi_0    pypi
    pyyaml                    6.0                      pypi_0    pypi
    pyzmq                     25.0.1                   pypi_0    pypi
    qtconsole                 5.4.1                    pypi_0    pypi
    qtpy                      2.3.0                    pypi_0    pypi
    readline                  8.2                  h5eee18b_0  
    regex                     2022.10.31               pypi_0    pypi
    requests                  2.28.2                   pypi_0    pypi
    responses                 0.18.0                   pypi_0    pypi
    rfc3339-validator         0.1.4                    pypi_0    pypi
    rfc3986-validator         0.1.1                    pypi_0    pypi
    send2trash                1.8.0                    pypi_0    pypi
    sentencepiece             0.1.97                   pypi_0    pypi
    setuptools                65.6.3           py39h06a4308_0  
    six                       1.16.0                   pypi_0    pypi
    sniffio                   1.3.0                    pypi_0    pypi
    soupsieve                 2.4                      pypi_0    pypi
    sqlite                    3.41.1               h5eee18b_0  
    sqlparse                  0.4.3                    pypi_0    pypi
    stack-data                0.6.2                    pypi_0    pypi
    sympy                     1.11.1                   pypi_0    pypi
    terminado                 0.17.1                   pypi_0    pypi
    tinycss2                  1.2.1                    pypi_0    pypi
    tk                        8.6.12               h1ccaba5_0  
    tokenizers                0.13.2                   pypi_0    pypi
    torch                     2.0.0                    pypi_0    pypi
    tornado                   6.2                      pypi_0    pypi
    tqdm                      4.65.0                   pypi_0    pypi
    traitlets                 5.9.0                    pypi_0    pypi
    transformers              4.27.0.dev0              pypi_0    pypi
    triton                    2.0.0                    pypi_0    pypi
    typing-extensions         4.5.0                    pypi_0    pypi
    tzdata                    2022g                h04d1e81_0  
    uri-template              1.2.0                    pypi_0    pypi
    urllib3                   1.26.15                  pypi_0    pypi
    wcwidth                   0.2.6                    pypi_0    pypi
    webcolors                 1.12                     pypi_0    pypi
    webencodings              0.5.1                    pypi_0    pypi
    websocket-client          1.5.1                    pypi_0    pypi
    wheel                     0.38.4           py39h06a4308_0  
    widgetsnbextension        4.0.5                    pypi_0    pypi
    xxhash                    3.2.0                    pypi_0    pypi
    xz                        5.2.10               h5eee18b_1  
    yarl                      1.8.2                    pypi_0    pypi
    zipp                      3.15.0                   pypi_0    pypi
    zlib                      1.2.13               h5eee18b_0  

    Package                  Version
    ------------------------ -----------
    accelerate               0.17.1
    aiohttp                  3.8.4
    aiosignal                1.3.1
    anyio                    3.6.2
    argon2-cffi              21.3.0
    argon2-cffi-bindings     21.2.0
    arrow                    1.2.3
    asgiref                  3.6.0
    asttokens                2.2.1
    async-timeout            4.0.2
    attrs                    22.2.0
    backcall                 0.2.0
    beautifulsoup4           4.11.2
    bitsandbytes             0.37.1
    bleach                   6.0.0
    certifi                  2022.12.7
    cffi                     1.15.1
    charset-normalizer       3.1.0
    cmake                    3.26.0
    comm                     0.1.2
    datasets                 2.10.1
    debugpy                  1.6.6
    decorator                5.1.1
    defusedxml               0.7.1
    dill                     0.3.6
    Django                   4.1.7
    executing                1.2.0
    fastjsonschema           2.16.3
    filelock                 3.10.0
    fqdn                     1.5.1
    frozenlist               1.3.3
    fsspec                   2023.3.0
    huggingface-hub          0.13.2
    idna                     3.4
    importlib-metadata       6.0.0
    ipykernel                6.21.3
    ipython                  8.11.0
    ipython-genutils         0.2.0
    ipywidgets               8.0.4
    isoduration              20.11.0
    jedi                     0.18.2
    Jinja2                   3.1.2
    jsonpointer              2.3
    jsonschema               4.17.3
    jupyter                  1.0.0
    jupyter_client           8.0.3
    jupyter-console          6.6.3
    jupyter_core             5.2.0
    jupyter-events           0.6.3
    jupyter_server           2.4.0
    jupyter_server_terminals 0.4.4
    jupyterlab-pygments      0.2.2
    jupyterlab-widgets       3.0.5
    lit                      15.0.7
    loralib                  0.1.1
    MarkupSafe               2.1.2
    matplotlib-inline        0.1.6
    mistune                  2.0.5
    mpmath                   1.3.0
    multidict                6.0.4
    multiprocess             0.70.14
    nbclassic                0.5.3
    nbclient                 0.7.2
    nbconvert                7.2.10
    nbformat                 5.7.3
    nest-asyncio             1.5.6
    networkx                 3.0
    notebook                 6.5.3
    notebook_shim            0.2.2
    numpy                    1.24.2
    nvidia-cublas-cu11       11.10.3.66
    nvidia-cuda-cupti-cu11   11.7.101
    nvidia-cuda-nvrtc-cu11   11.7.99
    nvidia-cuda-runtime-cu11 11.7.99
    nvidia-cudnn-cu11        8.5.0.96
    nvidia-cufft-cu11        10.9.0.58
    nvidia-curand-cu11       10.2.10.91
    nvidia-cusolver-cu11     11.4.0.1
    nvidia-cusparse-cu11     11.7.4.91
    nvidia-nccl-cu11         2.14.3
    nvidia-nvtx-cu11         11.7.91
    packaging                23.0
    pandas                   1.5.3
    pandocfilters            1.5.0
    parso                    0.8.3
    peft                     0.3.0.dev0
    pexpect                  4.8.0
    pickleshare              0.7.5
    pip                      23.0.1
    platformdirs             3.1.1
    prometheus-client        0.16.0
    prompt-toolkit           3.0.38
    psutil                   5.9.4
    ptyprocess               0.7.0
    pure-eval                0.2.2
    pyarrow                  11.0.0
    pycparser                2.21
    Pygments                 2.14.0
    pyrsistent               0.19.3
    python-dateutil          2.8.2
    python-json-logger       2.0.7
    pytz                     2022.7.1
    PyYAML                   6.0
    pyzmq                    25.0.1
    qtconsole                5.4.1
    QtPy                     2.3.0
    regex                    2022.10.31
    requests                 2.28.2
    responses                0.18.0
    rfc3339-validator        0.1.4
    rfc3986-validator        0.1.1
    Send2Trash               1.8.0
    sentencepiece            0.1.97
    setuptools               65.6.3
    six                      1.16.0
    sniffio                  1.3.0
    soupsieve                2.4
    sqlparse                 0.4.3
    stack-data               0.6.2
    sympy                    1.11.1
    terminado                0.17.1
    tinycss2                 1.2.1
    tokenizers               0.13.2
    torch                    2.0.0
    tornado                  6.2
    tqdm                     4.65.0
    traitlets                5.9.0
    transformers             4.27.0.dev0
    triton                   2.0.0
    typing_extensions        4.5.0
    uri-template             1.2.0
    urllib3                  1.26.15
    wcwidth                  0.2.6
    webcolors                1.12
    webencodings             0.5.1
    websocket-client         1.5.1
    wheel                    0.38.4
    widgetsnbextension       4.0.5
    xxhash                   3.2.0
    yarl                     1.8.2
    zipp                     3.15.0