therne / dmn-tensorflow

Dynamic Memory Networks (https://arxiv.org/abs/1603.01417) in Tensorflow

👋 I'm Jun- currently building Ace, AI project manager that gives teams perfect memory by auto-collecting & documenting recent work history.

My life story TL;DR

• 🖥️ started code at my age 9
• 💿 made my own OS at 12 because I was frustrated by Windows Me
• 👥 built a tech community with 200K builders at 14 because I wanted a place to show off my product and hang out with users
• 🚀 joined Airbridge as employee #1. left at team size 150, series B
• 🎵 launched a KPOP girl group tripleS. Built Cosmo as a founding engineer

dmn-tensorflow's Issues

train on default setting on qa1 but accuracy stay about 20%

Error in op

I run into the following problem. My env Python 3.6, Tensorflow 0.12

WARNING:tensorflow:From ./main.py:79 in main.: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02.
Instructions for updating:
Use tf.global_variables_initializer instead.
Traceback (most recent call last):
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1021, in _do_call
return fn(*args)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1003, in _run_fn
status, run_metadata)
File "anaconda3/lib/python3.6/contextlib.py", line 89, in exit
next(self.gen)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 469, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'DMN/Placeholder' with dtype bool
[[Node: DMN/Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "./main.py", line 89, in
tf.app.run()
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "./main.py", line 79, in main
sess.run(tf.initialize_all_variables())
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: You must feed a value for placeholder tensor 'DMN/Placeholder' with dtype bool
[[Node: DMN/Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]]

Caused by op 'DMN/Placeholder', defined at:
File "./main.py", line 89, in
tf.app.run()
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "./main.py", line 78, in main
model = DMN(FLAGS, words)
File "temp/dmn/dmn-tensorflow/models/base_model.py", line 18, in init
self.build()
File "temp/dmn/dmn-tensorflow/models/new/dmn_plus.py", line 25, in build
is_training = tf.placeholder(tf.bool)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1587, in placeholder
name=name)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2043, in _placeholder
name=name)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2240, in create_op
original_op=self._default_original_op, op_def=op_def)
File "anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1128, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'DMN/Placeholder' with dtype bool
[[Node: DMN/Placeholder = Placeholderdtype=DT_BOOL, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]]

python keyword "type" and build in function "input()" is overwritten.

Variable not found when initializing

First of all, i am excited to see this awesome implementation, your code is easy to read !

ValueError: Variable DMN/Episodic/Layer0/AttnGate/AttrGRU/Gates/Linear1/BatchNorm/DMN/Episodic/Layer0/AttnGate/AttrGRU_1/Gates/Linear1/BatchNorm/cond/DMN/Episodic/Layer0/AttnGate/AttrGRU_1/Gates/Linear1/BatchNorm/moments/normalize/mean/ExponentialMovingAverage/biased does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

I am on python3.5
I have searched the code and there isn't any variable declared with the error-prone "tf.variable()"
All of them are declared with tf.getvariable()
So i am not sure what is the problem here . Maybe you can help me ? Thanks really a lot !
@therne

Result on 1k

Would you be able to report the result on the 1k dataset as well? DMN+ paper doesn't report the 1k result so I was always wondering.. Thanks for great work!

ValueError: Variable DMN/Episodic/Layer0/...BatchNorm/moments/Squeeze/ExponentialMovingAverage/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

Please how to solve this problem. It seems complicated.

ValueError: Variable DMN/Episodic/Layer0/AttnGate/AttrGRU/Gates/Linear1/BatchNorm/DMN/Episodic/Layer0/AttnGate/AttrGRU_1/Gates/Linear1/BatchNorm/moments/Squeeze/ExponentialMovingAverage/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

Can you explain bit about input module of the dynamic memory network(DMN+) mentioned in here.

In the input module how to embed the information that coming from the context? In some papers it has mentioned

The will concatenate all the words in the context and add an EOS at the end of each sentence and feed in through r RNN with GRU units. Then take the hidden states at each time step.
Ask Me Anything:Dynamic Memory Networks for Natural Language Processing

In cases where the input sequence is a list of sentences,
we concatenate the sentences into a long list of word
tokens, inserting after each sentence an end-of-sentence token.

I went through the code it's bit different here. What is actually happening ? Each sentence separately feed through a RNN ?

Some error while running

Hi, I simply run the code by ./main.py --task 1 and error occurred.

Traceback (most recent call last):
  File "./main.py", line 7, in <module>
    from read_data import read_babi, get_max_sizes
  File "/home/james/Work/dmn-tensorflow/read_data.py", line 76
    word_table.add_vocab(*q, x["A"])    
SyntaxError: only named arguments may follow *expression

Is seems that something is wrong with data loader.

Old version of tensorflow

Hi!

It looks like you're using tensorflow 0.8 and not 1.0
https://www.tensorflow.org/install/migration
Your code does not work with tensorflow v. 1.0! Just saying ;)
I suggest you to modify your readme's prerequisites (or your code if you have the time :p)!

Thanks for your code and your work! :D

Input module in the dmn+

In the input module how to embed the information that coming from the context? In some papers it has mentioned

The will concatenate all the words in the context and add an EOS at the end of each sentence and feed in through r RNN with GRU units. Then take the hidden states at each time step.

I went through the code it's bit different here. What is actually happening ? Each sentence separately feed through a RNN ?

On python3.5, running dmn model I come to this error

TypeError: Input 'y' of 'Equal' Op has type int64 that does not match type int32 of argument 'x'.

softmax in def attention() is wrong?

Hi, first of all, thank you very much for the code, I learnt a lot!

In models/new/episode_module.py, line 59, l2 = tf.nn.softmax(l2) is performed on batch/N dimension but in paper https://arxiv.org/pdf/1603.01417v1.pdf formula (10), the softmax should be performed on Fact/F dimension. Is it the case?

Please let me know. Thank you!

Best regards,