borelien / entailment-neural-attention-lstm-tf

(arXiv:1509.06664) Reasoning about Entailment with Neural Attention.

Reasoning about Entailment with Neural Attention

This is a TensorFlow [3] implementation of the model described in Rocktäschel et al. "Reasoning about Entailment with Neural Attention" [1].

Data

The Stanford Natural Language Inference (SNLI) Corpus

The SNLI dataset by Samuel R. Bowman et al. [4]:

http://nlp.stanford.edu/projects/snli/snli_1.0.zip

Word2Vect

The pretrained Word2Vec word and phrase vectors by Mikolov et al. [2]:

https://docs.google.com/uc?id=0B7XkCwpI5KDYNlNUTTlSS21pQmM&export=download

Instructions

The main script come with several options, which can be listed with the --help flag.

python main.py --help

To run the training:

python main.py --train

By default, the script runs on GPU 0 with these parameters values:

learning_rate = 0.001
weight_decay = 0.
batch_size_train = 24
num_epochs = 45
sequence_length = 20
embedding_dim = 300
num_units = 100

Results

(to be updated)

Notes

(to be updated)

References

[1] Tim Rocktäschel, Edward Grefenstette, Karl Moritz Hermann, Tomáš Kočiský, Phil Blunsom, Reasoning about Entailment with Neural Attention, 2015.

[2] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, Jeffrey Dean, Distributed Representations of Words and Phrases and their Compositionality, 2013.

[3] Google, Large-Scale Machine Learning on Heterogeneous Systems, 2015.

[4] Samuel R. Bowman, Gabor Angeli, Christopher Potts, Christopher D. Manning, The Stanford Natural Language Processing Group, A large annotated corpus for learning natural language inference, 2015.

entailment-neural-attention-lstm-tf's Issues

Reproduced the results?

Would you please clarify if the implementation reproduces the results reported in the paper or not? Thanks,

Documentation

For something this complex that other people might want to use this is criminally underdocumented.
It would be nice if I didn't have to go through the entire code to figure out how the trained model should be used.
You are also missing a requirements.txt, so who knows whether this is going to work a couple of years from now, not to mention the fact that I have no idea what dependencies the code has in the first place.
If you are still maintaining the project I would really appreciate at least a user guide, but some docstrings would also be more than welcome.

Need to fix error from deprecation warning in loading gensim word2vec model

h = o * tf.tanh(x=self.c[-1]) in line no. 73 in network.py

Hi,

In network.py, (line no 73)

h = o*tf.tanh(x=self.c[-1])

is correct? In the paper, h is derived by o and c which have the same timestamp, t.
It should be changed to h = o*tf.tanh(c)?