knhuq / dynamic-tensorflow-tutorial

Tensorflow tutorial of building different dynamic recurrent neural network

Tensorflow tutorial to build any model from scratch.

In this repo these are presented

1. Dynamic Vanilla RNN ---> Notebook , Code
2. Dynamic GRU ---> Notebook , Code
3. Dynamic LSTM ---> Notebook , Code
4. Dynamic 2layerStacked LSTM ---> Notebook , Code
5. Dynamic BiDirectional LSTM ---> Notebook , Code
6. Tensorboard Example --->Notebook
7. Tensorflow LSTM implementation with dynamic batch--->Notebook

These RNN, GRU, LSTM and 2layer Stacked LSTM is implemented with 8 by 8 MNIST dataset for checking.

This repository contains the simple example of dynamic seqence and batch vhanilla RNN,GRU, LSTM,2layer Stacked LSTM, BiDirectional LSTM written in tensorflow using scan and map ops.

Every folder contains with .python file and ipython notebook for convenience.

This examples gives a very good understanding of the implementation of Dynamic RNN in tensorflow. These code can be extended to create neural stack machine, neural turing machine, RNN-EMM in tensorflow.

For any questions please ask them on twitter and I will love to answer those. Follow me on twitter for more updates.

dynamic-tensorflow-tutorial's Issues

Can not run your repo in tensorflow 1.3.0

Hi, KnHuq.
Thank you for the tutorial, and I've learned a lot about tensorflow from your code. But now, I am having a problem when I run your code and always report an error: TypeError: Singleton array array(<map object at 0x7fe600c28d30>, dtype=object) cannot be considered a valid collection.
I have replaced the "from sklearn.cross_validation import train_test_split" with "from sklearn.model_selection import train_test_split",but it is still an error.
I have been troubled by this problem for a long time,could you help me figure out the reason?
look forward to your reply

what is the difference between your dynamic implemetion of rnn and tf.nn.dynamic_rnn function ?

https://www.tensorflow.org/versions/r0.10/api_docs/python/nn.html#dynamic_rnn

process_batch_input_for_RNN could be shortened?

    batch_input_ = tf.transpose(batch_input, perm=[2, 0, 1])
    X = tf.transpose(batch_input_)

should be equivalent to

    X = tf.transpose(batch_input, perm=[1, 0, 2])

AttributeError: 'module' object has not attribute 'scan'

Hi,

I am trying to run the examples provided. However, it complains that tf module has no object 'scan'. I am using Tensorflow version 0.7.1 installed via pip for Python 2.7

Please help with Neural Stack Machine

Hi
Thank you very much for your tutorials. Please could you roughly tell how we can implemented the augmented memory module i.e. the neural stack of the Neural Stack Machine in tensorflow. I have understood the theory and I know how CNNs/RNNs/Autoencoders work in tensorflow but I'm unable to translate the augmented memory aspect into tensorflow.
Please help
Thank you

Can you explain why batch_input size ?

Hello,

I am a beginner to neural networks. Can you explain why
batch_input_ = tf.transpose(batch_input, perm=[2, 0, 1])
X = tf.transpose(batch_input_) ?
I know this is equivalent to

X = tf.transpose(batch_input, perm = [1,0,2]

but why do we need to reshape batch_input size and can you explain more about batch input size?

Thank you

Run your repo in tensorflow 1.1.0

Hi KnHuq,
Thanks for your code! I can learn a lot about the tensorflow usage. But when I ran your code in newer tf version, like 1.1.0, some functions need to be changed, e.g. tf.pack and tf.unpack.
After the modifications, I still encounter the problem "axis 0 specified more than once" at this line of code "sess.run(train_step, feed_dict={rnn._inputs: X, y: Y})".
I was stopped by this problem for two days, currently I don't know what I can do. can you help me figure out the reason?