Some Issues Regarding Code Replication about nemo-skills

[seamoke]

Sounds good! By the way, if you can share - what's the reason why you prefer huggingface's trainer to the provided finetuning workflow that's using nemo?

just I have some other code in that repo. So I use Trainer to finetune.

[Kipok]

@seamoke can you please add a couple of details to make it easier for us to help you.

• Did you use the latest commit or v0.1 tag (the sft container was upgraded to the newer nemo version and it might have some changes to the training results, although it shouldn't be this drastic)?
• To double check - you're using our provided config https://github.com/Kipok/NeMo-Skills/blob/main/nemo_skills/finetuning/sft_config.yaml and only replace epoch/lr, but other parameters are unchanged?
• Can you also share the plot of the validation accuracy (should be enabled by default, unless you changed that)?

[Kipok]

@seamoke are you still having issues? Going to close the issue if we don't get a reply from you

[seamoke]

@seamoke are you still having issues? Going to close the issue if we don't get a reply from you

Thanks for your reply. Because I use huggingface's Trainer to finetune model. So it's hard to compare the difference. I will close the issue and check my code again.

[Kipok]

Sounds good! By the way, if you can share - what's the reason why you prefer huggingface's trainer to the provided finetuning workflow that's using nemo?