How to costumize my own Transformer model? about deeplearningexamples

[jbaczek]

Transformer model inherits from FairseqModel. It is implemented in fairseq/models/transformer.py. Your custom model has to inherit from FairseqModel (which is defined in fairseq/models/fairseq_model.py. You have to implement build_model function which is called in taksk.build_model function. Next you have to register model with decorator @register_model. For any for predefined configurations of your custom model use decorator @register_model_architecture. I believe that after reading these two files everything will be clear.

[yaoyiran]

@jbaczek Thanks for your answer! A follow-up question: if I wrote my own model e.g. fairseq/models/mymodel.py or I directly modify and overwrite fairseq/models/transformer.py, do I need to run “pip install -e . ” or "python setup.py install" again? Does fairseq/models/mymodel.py rely on setup.py?

[jbaczek]

No, setup.py builds only C++ and CUDA extensions. Your code should work without rebuilding the fairseq.