Use of decapoda-research/llama-7b-hf checkpoint for the LLaMa-7B about alpaca_farm

[lxuechen]

Thanks for opening this issue!

The released checkpoints are weight diffs between our models and the Hugging Face checkpoint converted from Meta's released checkpoints based on newer versions of the transformers library (4.29.2).

The decapoda-research/llama-7b-hf checkpoint is converted with transformers version 4.27.0.dev0, so there could be a mismatch in the layout of the weights. Note the code for the llama model in the transformers library has undergone substantial changes since March.

We recommend converting Meta's released weights with the latest transformers library. I will improve the documentation on this. Closing this issue for now, but feel free to reopen if it still doesn't work after re-conversion.

[Yuhuajoe]

Hi, i have a similar questions , when i recover alpaca-farm-ppo-sim model, i got a different model_sum number(50837.671875), which should be equal to 50835.43359375 as expected.
I converted llama-7b-hf model from Meta's released checkpoints based on transformers==4.29.2 and i got correct result for sft-10k model after recovering model weights.
I'm wondering which reasons may cause this problem?

[lxuechen]

Hi, i have a similar questions , when i recover alpaca-farm-ppo-sim model, i got a different model_sum number(50837.671875), which should be equal to 50835.43359375 as expected.

Could you clarify which base model you're performing the diff recovery with? To be clear, we wouldn't expect the same result using decapoda-research/llama-7b-hf vs a freshly converted llama-7b-hf model from Meta's released checkpoints.

The reason being that decapoda-research/llama-7b-hf is based on an older version of transformers, and the model def'n of llama has changed dramatically in recent months.

[Yuhuajoe]

does that mean small difference is reasonable and i can trust the performance with recovered model?

[Yuhuajoe]

The base model i used was converted from Meta's released checkpoints via transformers==4.29.2.

[lxuechen]

The base model i used was converted from Meta's released checkpoints via transformers==4.29.2.

Thanks for following up. Just to confirm, did you run our recovery script as it is (w/o modifications)?

[Yuhuajoe]

Actually, there is a bit different. I download the wdiff from huggingface in advance, then recover as your recovery script do since some network problem would occur when auto-download via transformers' from_pretrained func.

[rtaori]

I see, very interesting. I will take a closer look at this next week to figure out what's going on. Our cluster is down for maintenance this week, apologize for the delay here, so I'll come back with an update next week once I can access the model checkpoints again.

[rtaori]

Hi @Yuhuajoe,
I am following up on this. I ran the following command

python -m pretrained_models.recover_model_weights \
  --llama-7b-hf-dir /juice5/scr5/nlp/llama_model/llama_hf_latest/llama-7b \
  --alpaca-farm-model-name ppo-sim \
  --models-save-dir /juice5/scr5/rtaori/tmp_alpaca_farm/

And I was able to reconstruct the ppo-sim weights with a matching model_sum.txt (no error in the script).

Can I suggest that you do the following again:

1. Re-convert the llama checkpoint with transformers==4.29.2,
2. Run the above command to auto-reconstruct the model weights.

The bug you're seeing could be either due to incorrect conversion or some issue in the wdiff pre-download you mentioned.

If you're still seeing an issue after this, please feel free to re-open this issue or create another issue post with the relevant details.

[bpucla]

I also get the integrity_check error with sft10k.
The base ckpt was a raw Meta ckpt converted with transformer=4.34.0
Any insights would be appreciated!

The command is

  python -m pretrained_models.recover_model_weights \
  --llama-7b-hf-dir <path1> \
  --alpaca-farm-model-name sft10k \
  --models-save-dir <path2>