Release pretrained models for BERT base? about bert_on_stilts

[zphang]

We retrained the models on STILTs in preparation for the code/model release, so we prioritized the BERT Large models that represented our headline results. In addition, BERT Large models are relatively harder for users to train because of significantly large GPU memory requirement, whereas BERT Base is relatively reasonable to train on consumer-grade GPUs. Conversely, running the BERT Large models in inference should be feasible for most users.

We will consider releases for BERT Base as well as adapter layers (https://arxiv.org/abs/1902.00751) in the near future.