Test ULMFit on CLS (Cross Lingual Sentiment) about multifit

[PiotrCzapla]

@eisenjulian if you don't mind let's try to keep the same structure for the datasets. That is:

• data/<dataset_name>/<lang>-<size+other info>/
  • <lang>.train.csv
  • <lang>.test.csv
  • <lang>.unsup.csv
  • <lang>.dev.csv

Then adding a new classifier to work with the scripts would be as easy as adding this 3 lines to configure it:
https://github.com/n-waves/ulmfit-multilingual/blob/master/ulmfit/train_clas.py#L117-L119

[eisenjulian]

Will do, thanks for the reference

[eisenjulian]

Quick update on this, performance seems to be quite poor just now, on the 70% for accuracy (it's binary classification) with balanced dataset.
This is the setup I'm using

git checkout cls
python prepare_cls.py
python -m ulmfit cls --dataset-path data/cls/de-dvd  --base-lm-path data/wiki/de-100/models/sp30k/lstm_nl4.m --tokenizer='sp' --lang=de --name 'nl4' - train 20 --bs 40

In the past, with regular tokenization, I was getting closer to 87% so something is definitely odd

[eisenjulian]

Updated script to

python prepare_cls.py https://storage.googleapis.com/ulmfit/cls
python -m ulmfit cls --dataset-path data/cls/de-dvd  --base-lm-path data/wiki/de-100/models/sp30k/lstm_nl4.m --tokenizer='sp' --lang=de --name 'nl4' - train 20 --bs 40