xyangyan / metabert

Codes for "MetaBERT: Collaborative Meta-Learning for Accelerating BERT Inference", published in CSCWD 2023.

MetaBERT

Codes for "MetaBERT: Collaborative Meta-Learning for Accelerating BERT Inference", published in CSCWD 2023.

Requirements

We recommend using Anaconda for setting up the environment of experiments:

conda create -n metabert python=3.8.8
conda activate metabert
conda install pytorch==1.8.1 cudatoolkit=11.1 -c pytorch -c conda-forge
pip install -r requirements.txt

Downstream task datasets

The GLUE task datasets can be downloaded from the GLUE leaderboard.

Please see our paper for more details!

Contact

If you have any problems, raise an issue or contact Yangyan Xu.

Citation

If you find this repo helpful, we'd appreciate it a lot if you can cite the corresponding paper:

@inproceedings{xu2023metabert,
  title={MetaBERT: Collaborative Meta-Learning for Accelerating BERT Inference},
  author={Xu, Yangyan and Yuan, Fangfang and Cao, Cong and Zhang, Xiaoliang and Su, Majing and Wang, Dakui and Liu, Yanbing},
  booktitle={2023 26th International Conference on Computer Supported Cooperative Work in Design (CSCWD)},
  pages={119--124},
  year={2023},
  organization={IEEE}
}