afcarl / seq2seq.pytorch

Sequence-to-Sequence learning using PyTorch

Seq2Seq in PyTorch

This is a complete suite for training sequence-to-sequence models in PyTorch. It consists of several models and code to both train and infer using them.

Using this code you can train:

• Neural-machine-translation (NMT) models
• Language models
• Image to caption generation
• Skip-thought sentence representations
• And more...

Models

Models currently available:

• Simple Seq2Seq recurrent model
• Recurrent Seq2Seq with attentional decoder
• Google neural machine translation (GNMT) recurrent model
• Transformer - attention-only model from "Attention Is All You Need"
• ByteNet - convolution based encoder+decoder

Datasets

Datasets currently available:

• WMT16
• OpenSubtitles 2016
• COCO image captions

All datasets can be tokenized using 3 available segmentation methods:

• Character based segmentation
• Word based segmentation
• Byte-pair-encoding (BPE) as suggested by bpe with selectable number of tokens.

After choosing a tokenization method, a vocabulary will be generated and saved for future inference.

Training methods

The models can be trained using several methods:

• Basic Seq2Seq - given encoded sequence, generate (decode) output sequence. Training is done with teacher-forcing.
• Multi Seq2Seq - where several tasks (such as multiple languages) are trained simultaneously by using the data sequences as both input to the encoder and output for decoder.
• Image2Seq - used to train image to caption generators.

Usage

Example training scripts are available in scripts folder. Inference examples are available in examples folder.