yuhsinliu1993 / quora_questionpairs_dl

Kaggle Competition: Using deep learning to solve quora's question pairs problem

Kaggle Competition: Quora Question Pairs Problem

Only implement on a single model - ESIM

See infomation on https://www.kaggle.com/c/quora-question-pairs

Framework

References

1. A decomposable attention model for natural language inference (2016) proposed by Aparikh, Oscart, Dipanjand, Uszkoreit.

2. Reasoning about entailment with neural attention (2016) proposed by Tim Rockta schel.

3. Neural Machine Translation by Jointly Learning to Align and Translate (2016) proposed by Yoshua Bengio, Dzmitry Bahdanau, KyungHyun Cho.

4. Enhanced LSTM for Natural Language Inference (2017) proposed by Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, Diana Inkpen.

Prerequisites

• python 2.7 or 3+
• numpy
• Tensorflow
• Keras
• spaCy

Download spaCy pre-trained Glove embedding weights

# out-of-the-box: download best-matching default model
$ python -m spacy download en

# download best-matching version of specific model for your spaCy installation
$ python -m spacy download en_core_web_md

Usage

To clean the inputs data, and split them into training and validation data run:

$ bash clean.sh

To train a model on default settings: (epochs: 10, embedding size: 300, hidden units: 100, learning rate: 0.0004)

$ python run.py --mode=train --verbose --best_glove

To test a model:

$ python run.py --mode=eval

All options:

usage: run.py [-h] [--num_epochs NUM_EPOCHS] [--batch_size BATCH_SIZE]
              [--embedding_size EMBEDDING_SIZE] [--max_length MAX_LENGTH]
              [--seed SEED] [--input_data INPUT_DATA] [--test_data TEST_DATA]
              [--val_data VAL_DATA] [--num_classes NUM_CLASSES]
              [--num_hidden NUM_HIDDEN] [--num_unknown NUM_UNKNOWN]
              [--learning_rate LEARNING_RATE] [--keep_prob KEEP_PROB]
              [--best_glove] [--tree_truncate] [--verbose]
              [--load_model LOAD_MODEL] --mode MODE

optional arguments:
  -h, --help            show this help message and exit
  --num_epochs NUM_EPOCHS
                        Specify number of epochs
  --batch_size BATCH_SIZE
                        Specify number of batch size
  --embedding_size EMBEDDING_SIZE
                        Specify embedding size
  --max_length MAX_LENGTH
                        Specify the max length of input sentence
  --seed SEED           Specify seed for randomization
  --input_data INPUT_DATA
                        Specify the location of input data
  --test_data TEST_DATA
                        Specify the location of test data
  --val_data VAL_DATA   Specify the location of test data
  --num_classes NUM_CLASSES
                        Specify the number of classes
  --num_hidden NUM_HIDDEN
                        Specify the number of hidden units in each rnn cell
  --num_unknown NUM_UNKNOWN
                        Specify the number of unknown words for putting in the
                        embedding matrix
  --learning_rate LEARNING_RATE
                        Specify dropout rate
  --keep_prob KEEP_PROB
                        Specify the rate (between 0 and 1) of the units that
                        will keep during training
  --best_glove          Glove: using light version or best-matching version
  --tree_truncate       Specify whether do tree_truncate or not
  --verbose             Verbose on training
  --load_model LOAD_MODEL
                        Locate the path of the model
  --mode MODE           Specify mode: train or eval or predict

quora_questionpairs_dl's Issues

[ERROR]AssertionError

when I use test data to predict ：
python run.py --mode=pred --load_model="model_dir/model.h5" --test_data="./data/test.csv"
I get this error:

Traceback (most recent call last):
  File "run.py", line 287, in <module>
    tf.app.run(main=run, argv=[sys.argv[0]] + unparsed)
  File "/Users/me/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "run.py", line 166, in run
    do_pred(FLAGS.test_data)
  File "run.py", line 38, in do_pred
    model = esim.build_model(FLAGS.load_model)
  File "/Users/me/iMac/400/403GithubProject/Quora_QuestionPairs_DL/ESIM.py", line 29, in build_model
    embedded_a = embedding_layer(a)
  File "/Users/me/iMac/400/403GithubProject/Quora_QuestionPairs_DL/layers.py", line 46, in __call__
    projected = self.project(self.embed(sentence))
  File "/Users/me/anaconda3/lib/python3.6/site-packages/keras/engine/topology.py", line 603, in __call__
    output = self.call(inputs, **kwargs)
  File "/Users/me/anaconda3/lib/python3.6/site-packages/keras/layers/wrappers.py", line 203, in call
    output_shape = self.compute_output_shape(input_shape)
  File "/Users/me/anaconda3/lib/python3.6/site-packages/keras/layers/wrappers.py", line 160, in compute_output_shape
    child_output_shape = self.layer.compute_output_shape(child_input_shape)
  File "/Users/me/anaconda3/lib/python3.6/site-packages/keras/layers/core.py", line 852, in compute_output_shape
    assert input_shape[-1]
AssertionError

the test data is Kaggle's test.csv

environment：
macos
python3.6
tensorflow 1.12.0
kesas 2.1.2