Failed to run main_demo_13B.py about chat-univi

[jpthu17]

This error is due to insufficient CPU RAM, specifically, the model parameters are loaded into CPU RAM before they are assigned to the GPUs. You can refer to the official documents: https://huggingface.co/blog/accelerate-large-models

Please try to use the option offload_folder="offload".

[wegigabyteinsn]

This error is due to insufficient CPU RAM, specifically, the model parameters are loaded into CPU RAM before they are assigned to the GPUs. You can refer to the official documents: https://huggingface.co/blog/accelerate-large-models

Please try to use the option offload_folder="offload".

Loading checkpoint shards: 67%|███████████████████████████████████████████████████████████████████████████████████████████████████▎ | 2/3 [00:18<00:09, 9.21s/it]Killed

[jpthu17]

I have not encountered these errors before, according to the instructions of the official documents, you can try to use the option offload_state_dict = True, torch_dtype=torch.float16.