abacinlp / investlm Goto Github PK

Can you realease the weights of model 7B or 13B for use on machines with limited hardware? 65B model is too heavy to handle

PS C:\Users\godsh\documents\github\investlm> python inference.py --base_model "Neko-Institute-of-Science/LLaMA-65B-HF" --lora_weights ""
use Linear Scaled RoPE...
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████| 14/14 [00:01<00:00, 8.17it/s]
WARNING:root:Some parameters are on the meta device device because they were offloaded to the disk.
Traceback (most recent call last):
File "C:\Users\godsh\documents\github\investlm\inference.py", line 106, in
fire.Fire(main)
File "C:\Users\godsh\AppData\Local\Programs\Python\Python311\Lib\site-packages\fire\core.py", line 141, in Fire
component_trace = _Fire(component, args, parsed_flag_args, context, name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godsh\AppData\Local\Programs\Python\Python311\Lib\site-packages\fire\core.py", line 475, in _Fire
component, remaining_args = _CallAndUpdateTrace(
^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godsh\AppData\Local\Programs\Python\Python311\Lib\site-packages\fire\core.py", line 691, in _CallAndUpdateTrace
component = fn(*varargs, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godsh\documents\github\investlm\inference.py", line 43, in main
model = PeftModel.from_pretrained(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\godsh\AppData\Local\Programs\Python\Python311\Lib\site-packages\peft\peft_model.py", line 324, in from_pretrained
PeftConfig._get_peft_type(
File "C:\Users\godsh\AppData\Local\Programs\Python\Python311\Lib\site-packages\peft\config.py", line 180, in _get_peft_type
if os.path.isfile(os.path.join(path, CONFIG_NAME)):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 108, in join
TypeError: expected str, bytes or os.PathLike object, not bool
PS C:\Users\godsh\documents\github\investlm>