how to make it run on Apple silicon(M1) on Mac about chatrwkv

[BlinkDL]

For your reference, here is a diff that I tested with Apple Silicon's GPU.

added MPS to rwkv 0.7.2 :)

[BlinkDL]

Firstly let's see if you can get https://github.com/BlinkDL/ChatRWKV/blob/main/RWKV_in_150_lines.py to work

I don't have Macbook so please try it

[ishikawa]

For your reference, here is a diff that I tested with Apple Silicon's GPU.

diff --git a/rwkv_pip_package/src/rwkv/model.py b/rwkv_pip_package/src/rwkv/model.py
index 31ef37b..45f2a29 100644
--- a/rwkv_pip_package/src/rwkv/model.py
+++ b/rwkv_pip_package/src/rwkv/model.py
@@ -79,7 +79,7 @@ class RWKV(MyModule):
         else:
             prxxx = lambda *args, **kwargs: None

-        STRATEGY_REGEX = r"^(?:(?:^|->) *(?:cuda(?::[\d]+)?|cpu) (?:fp(?:16|32)|bf16)(?:i8|i4|i3)?(?: \*[\d]+\+?)? *)+$"
+        STRATEGY_REGEX = r"^(?:(?:^|->) *(?:cuda(?::[\d]+)?|cpu|mps) (?:fp(?:16|32)|bf16)(?:i8|i4|i3)?(?: \*[\d]+\+?)? *)+$"
         if not re.match(STRATEGY_REGEX, strategy):
             raise ValueError("Invalid strategy. Please read https://pypi.org/project/rwkv/")

diff --git a/v2/chat.py b/v2/chat.py
index 3348732..0672080 100644
--- a/v2/chat.py
+++ b/v2/chat.py
@@ -42,7 +42,8 @@ torch.backends.cuda.matmul.allow_tf32 = True
 ########################################################################################################

 # args.strategy = 'cpu fp32'
-args.strategy = 'cuda fp16'
+args.strategy = 'mps fp32'
+# args.strategy = 'cuda fp16'
 # args.strategy = 'cuda:0 fp16 -> cuda:1 fp16'
 # args.strategy = 'cuda fp16i8 *10 -> cuda fp16'
 # args.strategy = 'cuda fp16i8'

Run on PyTorch 2.0.0

$ python
Python 3.9.14 (main, Oct 17 2022, 08:48:05) 
[Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.__version__
'2.0.0'

[eisneim]

@ishikawa thank you! with such simple change, it works!

@BlinkDL just another quest do you have plans to support Lora(low rank adaption) for fine tuning on small data sets?

[BlinkDL]

@BlinkDL just another quest do you have plans to support Lora(low rank adaption) for fine tuning on small data sets?

You can use https://github.com/Blealtan/RWKV-LM-LoRA