point-alpaca's Issues
How to support you for 13B
Hi there,
How do we support you to train 13B?
13B is coming for sure, larger versions - maybe. Consider supporting us if you want it done faster. :)
Please let me know what all is needed.
encounter exception running chat.py: PytorchStreamReader failed reading zip archive: failed finding central directory
Decryption works fine with no issue.
When running py chat.py, encounter following error:
Loading ./result...
gpu_count 1
The tokenizer class you load from this checkpoint is not the same type as the class this function is called from. It may result in unexpected tokenization.
The tokenizer class you load from this checkpoint is 'LLaMATokenizer'.
The class this function is called from is 'LlamaTokenizer'.
Loading checkpoint shards: 33%|██████████████████▎ | 1/3 [00:07<00:14, 7.28s/it]
Traceback (most recent call last):
File "/home/dsu/ai/xf/src/transformers/modeling_utils.py", line 415, in load_state_dict
return torch.load(checkpoint_file, map_location="cpu")
File "/home/dsu/p3/lib/python3.10/site-packages/torch/serialization.py", line 797, in load
with _open_zipfile_reader(opened_file) as opened_zipfile:
File "/home/dsu/p3/lib/python3.10/site-packages/torch/serialization.py", line 283, in __init__
super().__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dsu/ai/xf/src/transformers/modeling_utils.py", line 419, in load_state_dict
if f.read(7) == "version":
File "/usr/lib/python3.10/codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 128: invalid start byte
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/dsu/ai/palpaca/mychat.py", line 41, in <module>
load_model("./result")
File "/home/dsu/ai/palpaca/mychat.py", line 27, in load_model
model = transformers.LlamaForCausalLM.from_pretrained(
File "/home/dsu/ai/xf/src/transformers/modeling_utils.py", line 2709, in from_pretrained
) = cls._load_pretrained_model(
File "/home/dsu/ai/xf/src/transformers/modeling_utils.py", line 3023, in _load_pretrained_model
state_dict = load_state_dict(shard_file)
File "/home/dsu/ai/xf/src/transformers/modeling_utils.py", line 431, in load_state_dict
raise OSError(
OSError: Unable to load weights from pytorch checkpoint file for './result/pytorch_model-00002-of-00003.bin' at './result/pytorch_model-00002-of-00003.bin'. If you tried to load a PyTorch model from a TF 2.0 checkpoint, please set from_tf=True.
I'm running with latest transformer 4.28.0.dev0 (pulled the code today), which has LlamaTokenizer, hence is the message I got earlier regarding tokenizer class warning. Tried the specific transfomers in requirements.txt (git+https://github.com/zphang/transformers.git@68d640f7c368bcaaaecfc678f11908ebbd3d6176), got the same error.
torch version: 2.0.0.
Anyone encounters similar issue and suggestion to resolve the issue? Thanks.
Retrain for 13B+
Would it be possible for you guys to do this?
Question about fine-tuning LLaMa
I was wondering if we could fine-tune LLaMa with our own training data and then apply this to transform it into Alpaca and it would work, or would it be better to fine-tune Alpaca directly? Is it possible at all?
Any plans to...?
https://github.com/nomic-ai/gpt4all Any plans to train llama on this dataset? That would be great.
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 while running on an RTX 3060 12GB, using 8-bit.
After loading the 8bit model I am facing the following issue:
Loading checkpoint shards: 100%|████████████████████████████████████████████████████████████████| 3/3 [00:14<00:00, [28/1000$
Human: asd
/home/sadmin/miniconda3/envs/pa/lib/python3.10/site-packages/transformers/generation/utils.py:1201: UserWarning: You have modi
fied the pretrained model configuration to control generation. This is a deprecated strategy to control generation and will be
removed soon, in a future version. Please use a generation configuration file (see https://huggingface.co/docs/transformers/m
ain_classes/text_generation)
warnings.warn(
Traceback (most recent call last):
File "/home/sadmin/point-alpaca/chat.py", line 102, in <module>
go()
File "/home/sadmin/point-alpaca/chat.py", line 72, in go
generated_ids = generator(
File "/home/sadmin/miniconda3/envs/pa/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_contex
t
return func(*args, **kwargs)
File "/home/sadmin/miniconda3/envs/pa/lib/python3.10/site-packages/transformers/generation/utils.py", line 1452, in generate
return self.sample(
File "/home/sadmin/miniconda3/envs/pa/lib/python3.10/site-packages/transformers/generation/utils.py", line 2504, in sample
next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
What I tried so far:
quantization_config = BitsAndBytesConfig(
llm_int8_threshold=1.0,
)
as a variable, then
quantization_config=quantization_config
to model = transformers.LLaMAForCausalLM.from_pretrained([...]).cuda{}
Also tried to just passllm_int8_threshold=1.0
to the loader - both ways it loads the model, but at generation I have another error:
return self._apply(lambda t: t.cuda(device))
NotImplementedError: Cannot copy out of meta tensor; no data!
Hardware: RTX 3060 12GB, Ryzen 5700X, 24GB RAM
Error: Checksums do not match. The file may be corrupted
I get Error: Checksums do not match. The file may be corrupted
when attempting to decrypt the following files
tokenizer_config.json.a5f5efb2240276709a923b1404e08d93cc896fd1bd31fbe173e1e2789ea210ef.enc
and
special_tokens_map.json.4ad09c72922c015ba04f09eabebe38fb34ecb721ca712922c62038eaf2d0bc61.enc
However, I get the correct checksums for the two (encrypted) files above, as well as for consolidated.00.pth
. For the other decrypted files, the checksums are also correct.
However, the decrypted files tokenizer_config.json
and special_tokens_map.json
are empty.
Do you know what might be causing the issue? Thank you in advance.
How can I chat with characters locally?
As listed in https://alpaca.point.space/ by hosting the model myself? For example, if I would like to create a character locally, should I further finetune the model with data from, say Genshin Impact, or I can do this through prompting?
Can you share md5 of the files in encrypted folder?
Running the model on alpaca.cpp
I've got the decrypted model, however I am unable to run chat.py due to insufficient video memory. How would I go about running the model on alpaca.cpp?
Also, a question on size: given that the original 7B model is a single 13.5GB file, how come the decryption process produces 3 files adding up to 27GB? Thanks!
Latest Commit Gives Incorrect Decryption
The decrypt.py
file in the latest commit doesn't give the correct decryption according to MD5 checksum. However, commit 9ee0219
, which doesn't utilize multiple CPU cores, give the correct decryption.
load_model("./result")
There is a mistake on line 42 of the chat.py file.
It should be load_model("./result")
Error: Checksums do not match. The file may be corrupted.
I saw the closed issue about the same, but I just pulled the repo and used the scripts. I think if a brand new installation returns this problem, it might worth investigating.
Decrypting file encrypted/config.json.caf9cac32580e31af8254f66c5a070741d70b15a651721748189180325b7d5a8.enc with 5 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/generation_config.json.c5c8961ed243834883fb4e45e8850d3873d6100fde97817f59d275a90eba269d.enc with 5 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/pytorch_model-00001-of-00003.bin.90d2ab95a32aeb9362814d8b86db2af5454baab8ea3aa8230c271d6962abb9db.enc with 5 workers
Writing final chunks...
Error: Checksums do not match. The file may be corrupted.
Question about Alpaca maximum sequence length
LLaMA seems to have a maximum sequence length of 2048 as stated and written in its source code. However I see that's the token config as a maximum sequence length of 512 here. Does Alpaca have a smaller maximum sequence length than LLaMA? Is that possible?
Corrupted Model Weights
Hey,
I am trying to update the 7B model weights. For all weigths, except one, it works fine.
The corrupted one is: pytorch_model-00003-of-00003.bin.72bf4c96aa6b0c7b56b0336791960da9c75de324ea1131ea4bfc20fde41115c8.enc
Any idea why this particular one is not working?
Greetings
Question about random output
Decryption on several files with matching md5 fail
Model files are fine but a few of the .json files aren't working.
Decrypting file encrypted/added_tokens.json.75e3ca5df2973756aa612cb17246ef6020a68ff8d94671508987d373642f7a36.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/config.json.caf9cac32580e31af8254f66c5a070741d70b15a651721748189180325b7d5a8.enc with 31 workers
Writing final chunks...
Error: Checksums do not match. The file may be corrupted.
Decrypting file encrypted/generation_config.json.c5c8961ed243834883fb4e45e8850d3873d6100fde97817f59d275a90eba269d.enc with 31 workers
Writing final chunks...
Error: Checksums do not match. The file may be corrupted.
Decrypting file encrypted/pytorch_model-00001-of-00003.bin.90d2ab95a32aeb9362814d8b86db2af5454baab8ea3aa8230c271d6962abb9db.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/pytorch_model-00002-of-00003.bin.f3c10a4f5c8beafc6667d34557b64ba479e4dde6ef10672287857b329b7e3229.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/pytorch_model-00003-of-00003.bin.72bf4c96aa6b0c7b56b0336791960da9c75de324ea1131ea4bfc20fde41115c8.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/pytorch_model.bin.index.json.07ca8edea996b6c3274395fdb2b6c9108f2ffdd610ae55e35c126c21a9d535b1.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/special_tokens_map.json.4ad09c72922c015ba04f09eabebe38fb34ecb721ca712922c62038eaf2d0bc61.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/tokenizer.model.9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/tokenizer_config.json.a5f5efb2240276709a923b1404e08d93cc896fd1bd31fbe173e1e2789ea210ef.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/trainer_state.json.43964ae247e74f4055fe1cf99a7a16efc3114402a1cd918b3cd9e2ebf2858ca9.enc with 31 workers
Writing final chunks...
Decryption completed.
Decrypting file encrypted/training_args.bin.02f8c3ba14e3c48c05f76880975d7385c878b0e5a0863e352c82f331150d2bd4.enc with 31 workers
Writing final chunks...
Decryption completed.
can the nice team of pointnetwork share some production details ?
Hi,
I think I just got banned from the telegram chat for a hosting related question regarding alpaca. Although it says:
Questions? Feedback? Chat with us in our Telegram community!
Sorry if I misunderstood the purpose of that chat so let me ask my question here again:
Can someone please share some insights on how to host the alpaca the way you did it ? I noticed that it is snappy and I would like to understand how you do it. Stack, optimizations, hardware ? Load balancing, dynamic batching ?
Thanks for any recommendations
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.