Question or bug about llama2lang

[ErikTromp]

The dataset is actually quite okay but we have had some issues with translation of a few languages so far, so we will have to redo a few too in the coming weeks.

As for oasst2: es it takes longer but you can already do that using the current scripts if you want to. Same for Mistral.

Mixtral we are not sure yet but will support in the future too.

[ErikTromp]

Yeah madlad was broken on special characters. I fixed this now, try again.

[IzzyHibbert]

Yeah madlad was broken on special characters. I fixed this now, try again.

Yes I confirm that it works.
For this translation steps, based on your experience, do you think that a 12Gb Vram could be enough ? (In colab I see it under 6Gb Vram at the moment - just started - so was wondering if I could rather translate on prem). Thanks

[ErikTromp]

You can always get it to work by lowering batch_size and/or using quantization but both slow down translation speeds unfortunately. We will be looking into adding a lot more translation models to pick from and making it easy to switch between those so hopefully in the (near) future this will become easier.

For now I would say that if your (target) language is well supported in OPUS (meaning from at least English and Spanish, there is a direct model, check HF) you should go with those. In other cases you have to opt for madlad but tweak the sizing and quantization (--madlad_quant for 8bit and --madlad_quant4 for 4bit).

[IzzyHibbert]

hat if your (target) language is well supported in OPUS (meaning from at least English and Spanish, there is a direct model, check HF) you should go with those.

Thanks for the details.
As for "translating the Dataset", Yes last week I made all the steps and use what you guys already made (big thanks!) all the way through without this initial training, but rather by reusing your QLora. The thing is that this morning I had a chance to do some tests with the resulting model and it was not really good. I guess that this might depend on the Dataset, rather than the translation (actually I saw that there is a bigger version of the same OpenAssistant : oasst2 but probably this requires much more translation time).

Last question: If I use another base model, let's say one that is based on Mistral 7b, I guess it's as simple as changing the base model arguments and no other attention points ?

(apologies for been off topic, ticket is fine and about to close it)