Not enough memory for 1 gpu about chatrwkv

[BlinkDL]

Run Single Model on Multiple GPUs:
#12
https://github.com/mpskex/ChatRWKV

[HaishengLiang]

3B model need 6GiB GPU memory or more.
7B model need 14GiB GPU memory or more.
14B model need 28GiB GPU memory or more.
Or you can run it with CPU by 128G memory limit.

reasoning of multiple graphics cards has not been implemented yet
you can use DeepSpeed for multiple graphics cards train

[HaishengLiang]

May be you can run model with 8bit not for bf16
it should be use less GPU memory.

[xiongsp]

Thanks. By the way, do you plan to implement reasoning of multiple graphics cards?

[xiongsp]

I see. Thanks!