Comments (10)
Thanks for opening this issue!
The released checkpoints are weight diffs between our models and the Hugging Face checkpoint converted from Meta's released checkpoints based on newer versions of the transformers library (4.29.2).
The decapoda-research/llama-7b-hf
checkpoint is converted with transformers version 4.27.0.dev0, so there could be a mismatch in the layout of the weights. Note the code for the llama model in the transformers library has undergone substantial changes since March.
We recommend converting Meta's released weights with the latest transformers library. I will improve the documentation on this. Closing this issue for now, but feel free to reopen if it still doesn't work after re-conversion.
from alpaca_farm.
Hi, i have a similar questions , when i recover alpaca-farm-ppo-sim model, i got a different model_sum number(50837.671875), which should be equal to 50835.43359375 as expected.
I converted llama-7b-hf model from Meta's released checkpoints based on transformers==4.29.2 and i got correct result for sft-10k model after recovering model weights.
I'm wondering which reasons may cause this problem?
from alpaca_farm.
Hi, i have a similar questions , when i recover alpaca-farm-ppo-sim model, i got a different model_sum number(50837.671875), which should be equal to 50835.43359375 as expected.
Could you clarify which base model you're performing the diff recovery with? To be clear, we wouldn't expect the same result using decapoda-research/llama-7b-hf
vs a freshly converted llama-7b-hf model from Meta's released checkpoints.
The reason being that decapoda-research/llama-7b-hf
is based on an older version of transformers, and the model def'n of llama has changed dramatically in recent months.
from alpaca_farm.
does that mean small difference is reasonable and i can trust the performance with recovered model?
from alpaca_farm.
The base model i used was converted from Meta's released checkpoints via transformers==4.29.2.
from alpaca_farm.
The base model i used was converted from Meta's released checkpoints via transformers==4.29.2.
Thanks for following up. Just to confirm, did you run our recovery script as it is (w/o modifications)?
from alpaca_farm.
Actually, there is a bit different. I download the wdiff from huggingface in advance, then recover as your recovery script do since some network problem would occur when auto-download via transformers' from_pretrained func.
from alpaca_farm.
I see, very interesting. I will take a closer look at this next week to figure out what's going on. Our cluster is down for maintenance this week, apologize for the delay here, so I'll come back with an update next week once I can access the model checkpoints again.
from alpaca_farm.
Hi @Yuhuajoe,
I am following up on this. I ran the following command
python -m pretrained_models.recover_model_weights \
--llama-7b-hf-dir /juice5/scr5/nlp/llama_model/llama_hf_latest/llama-7b \
--alpaca-farm-model-name ppo-sim \
--models-save-dir /juice5/scr5/rtaori/tmp_alpaca_farm/
And I was able to reconstruct the ppo-sim
weights with a matching model_sum.txt
(no error in the script).
Can I suggest that you do the following again:
- Re-convert the llama checkpoint with
transformers==4.29.2
, - Run the above command to auto-reconstruct the model weights.
The bug you're seeing could be either due to incorrect conversion or some issue in the wdiff pre-download you mentioned.
If you're still seeing an issue after this, please feel free to re-open this issue or create another issue post with the relevant details.
from alpaca_farm.
I also get the integrity_check
error with sft10k
.
The base ckpt was a raw Meta ckpt converted with transformer=4.34.0
Any insights would be appreciated!
The command is
python -m pretrained_models.recover_model_weights \
--llama-7b-hf-dir <path1> \
--alpaca-farm-model-name sft10k \
--models-save-dir <path2>
from alpaca_farm.
Related Issues (20)
- [Bug] Error importing is_deepspeed_zero3_enabled HOT 1
- Question about KL term HOT 1
- Reproducibility of pretuned reward model
- Use with Llama-2-70b-hf? HOT 4
- KeyError: 'llama' in /recover_model_weights.py HOT 2
- Confusing detail preference mapping HOT 1
- recover_model_weights.py gives WARNING:root:Your base LLaMA checkpoint is converted with transformers==4.27.0.dev0
- Huge memory demand of recover_model_weights.py?
- RewardModel.from_pretrained() loads redundant weights (incurs extra ~30GB of RAM)
- Why use FSDP instead of Deepspeed? HOT 2
- lower cuda version? HOT 1
- integrity_check error with sft10k
- score with reward model HOT 2
- RecursionError: maximum recursion depth exceeded
- BaseAnnotator.__init__() got an unexpected keyword argument 'other_keys_to_keep' HOT 4
- PairwiseAutoAnnotator always "Annotating 0 examples with gpt4_3" HOT 1
- Repeated Deprecation Error HOT 3
- Error downloading pre-trained weights
- Using pretrained models HOT 3
- Inquiry Regarding Supervised Fine-Tuning with AlpacaFarm Framework for Pythia Models HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from alpaca_farm.