Running PPO with fewer GPUs about alpaca_farm

[lxuechen]

Thanks for your interest! We haven't tested the current implementation on fewer than 8 GPUs for this particular instruction-following setup. But there are several things you may try

• low precision training, e.g., with bnb
• quantized training
• LoRA or other adapters

There are also recent approaches that combine several paradigms above such as QLoRA.

We're working on some code that would make this training much easier, but we don't have a timeline for that yet.

[Wallace-222]

May I know if there is any updates? Thanks a lot.

[rtaori]

Hi,
We don't have an updated timeline for this, sorry about that - it's not currently on our near-term roadmap. We'd be happy to accept any PRs that implement this, but until then closing this issue as it's getting stale.