PhasedLSTM-Keras

Keras implementation of Phased LSTM [https://arxiv.org/abs/1610.09513], from NIPS 2016.

Works both with Theano and Tensorflow backend (although Theano recommended, as 3x faster).

Classification performance compared to standard Keras LSTM for MNIST dataset:

Phased LSTM

Epoch 1/20 60000/60000 [==============================] - 324s - loss: 2.0418 - acc: 0.2513
Epoch 2/20 60000/60000 [==============================] - 319s - loss: 1.7242 - acc: 0.3654
Epoch 3/20 60000/60000 [==============================] - 314s - loss: 1.7099 - acc: 0.3501
Epoch 4/20 60000/60000 [==============================] - 314s - loss: 1.5299 - acc: 0.4254
Epoch 5/20 60000/60000 [==============================] - 313s - loss: 1.2343 - acc: 0.5388
Epoch 6/20 60000/60000 [==============================] - 314s - loss: 1.1064 - acc: 0.5926
Epoch 7/20 60000/60000 [==============================] - 314s - loss: 1.0078 - acc: 0.6425
Epoch 8/20 60000/60000 [==============================] - 314s - loss: 0.9120 - acc: 0.6825
Epoch 9/20 60000/60000 [==============================] - 314s - loss: 0.8294 - acc: 0.7134
Epoch 10/20 60000/60000 [==============================] - 311s - loss: 0.7552 - acc: 0.7434
Epoch 11/20 60000/60000 [==============================] - 311s - loss: 0.6813 - acc: 0.7685
Epoch 12/20 60000/60000 [==============================] - 311s - loss: 0.6143 - acc: 0.7901
Epoch 13/20 60000/60000 [==============================] - 311s - loss: 0.5686 - acc: 0.8028
Epoch 14/20 60000/60000 [==============================] - 311s - loss: 0.5320 - acc: 0.8156
Epoch 15/20 60000/60000 [==============================] - 311s - loss: 0.5097 - acc: 0.8223
Epoch 16/20 60000/60000 [==============================] - 311s - loss: 0.4750 - acc: 0.8353
Epoch 17/20 60000/60000 [==============================] - 311s - loss: 0.4507 - acc: 0.8467
Epoch 18/20 60000/60000 [==============================] - 312s - loss: 0.4354 - acc: 0.8538
Epoch 19/20 60000/60000 [==============================] - 316s - loss: 0.4106 - acc: 0.8618
Epoch 20/20 60000/60000 [==============================] - 316s - loss: 0.3934 - acc: 0.8695

LSTM

Epoch 1/20 60000/60000 [==============================] - 157s - loss: 2.2945 - acc: 0.1216
Epoch 2/20 60000/60000 [==============================] - 157s - loss: 2.0987 - acc: 0.2275
Epoch 3/20 60000/60000 [==============================] - 157s - loss: 1.9601 - acc: 0.2926
Epoch 4/20 60000/60000 [==============================] - 157s - loss: 1.8418 - acc: 0.3247
Epoch 5/20 60000/60000 [==============================] - 157s - loss: 2.0860 - acc: 0.2619
Epoch 6/20 60000/60000 [==============================] - 157s - loss: 2.1297 - acc: 0.2225
Epoch 7/20 60000/60000 [==============================] - 157s - loss: 1.8556 - acc: 0.3287
Epoch 8/20 60000/60000 [==============================] - 157s - loss: 1.8428 - acc: 0.3344
Epoch 9/20 60000/60000 [==============================] - 158s - loss: 1.8119 - acc: 0.3219
Epoch 10/20 60000/60000 [==============================] - 158s - loss: 1.8159 - acc: 0.3246
Epoch 11/20 60000/60000 [==============================] - 158s - loss: 1.9290 - acc: 0.2554
Epoch 12/20 60000/60000 [==============================] - 158s - loss: 1.7843 - acc: 0.3047
Epoch 13/20 60000/60000 [==============================] - 158s - loss: 1.7623 - acc: 0.3371
Epoch 14/20 60000/60000 [==============================] - 158s - loss: 1.6016 - acc: 0.4079
Epoch 15/20 60000/60000 [==============================] - 158s - loss: 1.5954 - acc: 0.3985
Epoch 16/20 60000/60000 [==============================] - 157s - loss: 1.6393 - acc: 0.3823
Epoch 17/20 60000/60000 [==============================] - 157s - loss: 1.6186 - acc: 0.3939
Epoch 18/20 60000/60000 [==============================] - 157s - loss: 1.6276 - acc: 0.3835
Epoch 19/20 60000/60000 [==============================] - 157s - loss: 1.6557 - acc: 0.3684
Epoch 20/20 60000/60000 [==============================] - 157s - loss: 1.8699 - acc: 0.3258`

stevenlol / phasedlstm-keras Goto Github PK