Any idea if this will work on CPU? about minigpt-4

[liyaozong1991]

if you want to run the demo on cpu, you need to use float32 and init all param on cpu.

1. change demo.py 
# model = model_cls.from_config(model_config).to('cuda:{}'.format(args.gpu_id)). 
model = model_cls.from_config(model_config).to('cpu')
# chat = Chat(model, vis_processor, device='cuda:{}'.format(args.gpu_id))
chat = Chat(model, vis_processor, device='cpu')
2. change minigpt4.yaml 
# vit_precision: "fp16"
vit_precision: "fp32"
3. change minigpt4_eval.yaml 
#  low_resource: True
low_resource: False
4. change mini_gpt4.py, about line 90:
#  torch_dtype=torch.float16,
torch_dtype=torch.float32

that is all you need to do to use cpu run the demo.py, but the speed is very slow 

[kenneth104]

yep,I need help to run on CPU

I am downloading and merging models

[webnizam]

Hi! Has there been any progress on running it on a CPU? I'm really interested in this as well, since I don't have a powerful GPU. Any updates or workarounds you've discovered would be greatly appreciated. Thanks!

[rdkmaster]

yep,I need help to run on CPU

I am downloading and merging models

@kenneth104
Any progress of using CPU was made? Can you share?

[kenneth104]

yep,I need help to run on CPU
I am downloading and merging models

@kenneth104 Any progress of using CPU was made? Can you share?

noI can’t run on CPU, something need cuda and output error

[rdkmaster]

@liyaozong1991 thanks, I'll try this.