lsheiba / tf2-transformer-chatbot

Transformer Chatbot in TensorFlow 2.1

Transformer Chatbot with TensorFlow 2.1

Build an end-to-end chatbot with Transformer in TensorFlow 2.1. Checkout the my tutorial on blog.tensorflow.org.

Updates

• 18 Jan 2019: Added notebook with Google Colab TPU support in TensorFlow 2.1.

Packages

• TensorFlow 2.0
• TensorFlow Datasets

pip install -r requirements.txt

Dataset

• We will use the conversations in movies and TV shows provided by Cornell Movie-Dialogs Corpus, which contains more than 220 thousands conversational exchanges between more than 10k pairs of movie characters, as our dataset.
• We pre-process our dataset in the following order:
  • Extract max_samples conversation pairs into list of questions and answers.
  • Pre-process each sentence by removing special characters in each sentence.
  • Build tokenizer (map text to ID and ID to text) using TensorFlow Datasets SubwordTextEncoder.
  • Tokenize each sentence and add start_token and end_token to indicate the start and end of each sentence.
  • Filter out sentence that has more than max_length tokens.
  • Pad tokenized sentences to max_length
• Check dataset.py implementation.

Model

• check model.py for the implementation of Multi-Headed Attention, Positional Encoding and Transformer.

Run

• check all availble flags and hyper-parameters python main.py --help

python main.py --batch_size 256 --epochs 50 --max_samples 50000

Samples

input:		where have you been?
output:		i m not talking about that .

input:		it's a trap!
output:		no , it s not .