test model on other languages about multifit

[PiotrCzapla]

Do you mean in form of zeroshoot transfer learning?

If so we use Laser for that. First we train laser , to obtain zeroshoot predictions for other languages.
Then we use that zershoot predictions to train regular multifit (pretrained in the language that we are testing on). The unsupervised pretraining removes noise from the laser zeroshoot predictions and improves the results.

[wonderfultina]

I understand now,thank you.

[vhargitai]

Hi @PiotrCzapla , have you or your colleagues already pretrained this model on English Wikipedia?

If not, would using prepare_wiki-en.sh to grab wikitext-103, then running postprocess_wikitext.py on it be identical to the dataset preparation you did for other languages in the MultiFiT paper?

I'd like to reproduce the monolingual supervised training procedure in the MultiFiT paper for English language classification. Thanks in advance!

[mhajiaghayi]

Do you mean in form of zeroshoot transfer learning?
If so we use Laser for that. First we train laser , to obtain zeroshoot predictions for other languages.
Then we use that zershoot predictions to train regular multifit (pretrained in the language that we are testing on). The unsupervised pretraining removes noise from the laser zeroshoot predictions and improves the results.

Q) in this case, you don't have a single model with the fixed tokenization that does zero-shot embedding for other language. am I right?

[iNeil77]

Do you mean in form of zeroshoot transfer learning?

If so we use Laser for that. First we train laser , to obtain zeroshoot predictions for other languages.
Then we use that zershoot predictions to train regular multifit (pretrained in the language that we are testing on). The unsupervised pretraining removes noise from the laser zeroshoot predictions and improves the results.

In the CLS-DE notebook I only see the classifier fine-tuning happening with DE Music Data, Label pairs. But if I understand what you said correctly, shouldn't the LASER classifier be fine-tuned with EN Music Data first before it can act as a teacher to fine-tune the DE Classifier? I don't see that in the notebook. Am I misunderstanding the training regime?