This is the code for training target language-ready (TLR) task adapters with torch
and adapter-transformers
. Please refer to our paper Cross Lingual Transfer with Target Language-Ready Task Adapters for background.
First, install python
>= 3.9 and pytorch
>= 1.12.1, e.g. using conda
:
conda create -n tlr-env python=3.9
conda activate tlr-env
conda install pytorch==1.12.1 pytorch-cuda=11.6 -c pytorch -c nvidia
Then download and install tlr-adapters
:
git clone https://github.com/parovicm/tlr-adapters
cd tlr-adapters
pip install -e .
The code for training TLR adapters is given by the TLR trainer
which modifies the original Trainer
from the adapter-transformers
.
Scripts for all tasks supported are provided in examples/
.
For example, the script for training the TASK-MULTI
variant (see the paper for the explanation) for the AmericasNLI
dataset is given here.
Generally, unless you are using a single language adapter during task adapter training (this is the case with MAD-X
and TARGET
variants) you need to pass the file containing language adapters to be used.
For the TASK_MULTI
variant of the AmericasNLI
dataset language adapters file will have the following content:
en PATH_TO_EN_LANG_ADAPTER
aym PATH_TO_AYM_LANG_ADAPTER
bzd PATH_TO_BZD_LANG_ADAPTER
cni PATH_TO_CNI_LANG_ADAPTER
gn PATH_TO_GN_LANG_ADAPTER
hch PATH_TO_HCH_LANG_ADAPTER
nah PATH_TO_NAH_LANG_ADAPTER
oto PATH_TO_OTO_LANG_ADAPTER
quy PATH_TO_QUY_LANG_ADAPTER
tar PATH_TO_TAR_LANG_ADAPTER
shp PATH_TO_SHP_LANG_ADAPTER
If you use this code, please cite the following paper:
Marinela Parović, Alan Ansell, Ivan Vulić, and Anna Korhonen. 2023. Cross Lingual Transfer with Target Language-Ready Task Adapters. In *Findings of the 61st Annual Meeting of the Association for Computational Linguistics.