lightonai / dfa-scales-to-modern-deep-learning Goto Github PK

I'm trying to run train_lm.py to replicate Table 5 in your paper. For the simple baseline experiment python train_lm.py --gpu_id 0 --beta2 0.98 I am missing the file bpe_models/wikitext-2.bpe.32000. How do I download or generate this file? Also the default --dataset is 'wikitext103', not 'wikitext2', is this compatible with bpe_models/wikitext-2.bpe.32000?

Thanks! And cool project!

Congratulations on your paper. I was trying to play around a bit and found that the dependencies are not easy to install (at least on arm64)

1. The requirements file is missing an = sign on line that refers to seaborn.
2. The package sentencepiece==1.9.0 is not available with pip on arm64.
3. torch==1.5.0+cu101 is not available with pip

The first two problems are easily fixable and the third one too, but having encountered three hiccups in installing just half of the dependencies listed in requirements.txt, I decided to file this issue. Please add better instructions for installing dependencies, or if possible, please provide docker images (arm64 compatible) to solve all hassles in one go.