thevasudevgupta / transformers-adapters Goto Github PK

Hi, thanks for your code. I used adapter_modeling.py and changed modeling_marian.py. But, the result is not good.
I used the pre model Helsinki-NLP/opus-mt-en-zh and datasets in thesis domain.
first experience,
enc_self_attn_adapter=True, dec_ffn_adapter=True, enc_tok_embed_adapter=True, dec_tok_embed_adapter=True,
sencend expericence,
enc_self_attn_adapter=True, dec_self_attn_adapter=True, cross_attn_adapter=True, enc_ffn_adapter=True, dec_ffn_adapter=True, enc_tok_embed_adapter=True, dec_tok_embed_adapter=True,
Both of them scored of bleu below baseline and much lower than fine-tuning.

Is this method only applicable to mbart? What do I need to pay attention to when I apply it to MarianMTModel?

Hi, thanks for this work, did you open-sourced the colab notebooks that were used to train this model!?