voslucas / smru

Softmax Recurrent Unit (SMRU) - A new type of gated RNN cell

Softmax Recurrent Unit (SMRU)

This repository contains the implementation and most of the experiments of the SMRU as reported in the paper Softmax Recurrent Unit: A new type of gated RNN cell by Lucas Vos and Twan van Laarhoven.

The paper is accepted and presented at ESANN 2020 proceedings, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Online event, 2-4 October 2020.

Reference

If you use this code or research, please cite the following :

@inproceedings{vos2020softmax,
  title={Softmax Recurrent Unit: A new type of RNN cell.},
  author={Vos, Lucas and van Laarhoven, Twan},
  booktitle={ESANN},
  pages={309--314},
  year={2020}
}

SMRU Implementation

We used an older PyTorch version, before the cuDNN implementations, to code the SMRU as a PyTorch compatible RNN cell. This implementation only provides a CPU based version of the SMRU.

Task runner

The task-manager.py provides a class for storing python commands to an Azure Blob Storage location, which can be run by remote machines running task-runner. The jobs can be stored/generated by task-runner.py.