ml-lab / rmn

Related Memory Network (RMN)

• End-to-End neural network architecture exploiting both memory network and relation network structures
• State-of-the-art result in jointly trained bAbI-10k story-based question answering

Result

Task	MemN2N	DMN+	RN	RMN
1	0.0	0.0	0.0	0.0
2	0.3	0.3	6.5	0.5
3	9.3	1.1	12.9	14.7
4	0.0	0.0	0.0	0.0
5	0.6	0.5	0.5	0.4
6	0.0	0.0	0.0	0.0
7	3.7	2.4	0.2	0.5
8	0.8	0.0	0.1	0.3
9	0.8	0.0	0.0	0.0
10	2.4	0.0	0.0	0.0
11	0.0	0.0	0.4	0.5
12	0.0	0.0	0.0	0.0
13	0.0	0.0	0.0	0.0
14	0.0	0.0	0.0	0.0
15	0.0	0.0	0.0	0.0
16	0.4	45.3	50.3	0.9
17	40.7	4.2	0.9	0.3
18	6.7	2.1	0.6	2.3
19	66.5	0.0	2.1	2.9
20	0.0	0.0	0.0	0.0
Mean error	6.6	2.8	3.7	1.2
Failed tasks	4	1	3	1

Prerequisites

• Python 3.6
• Tensorflow 1.3.0
• dependencies
  • pip install tqdm colorlog

Usage

1. prepare data

To process bAbI story-based QA dataset, run:

$ python preprocessor.py --data story

To process bAbI dialog dataset, run:

$ python preprocessor.py --data dialog

2. train model

To train RMN on bAbI story-based QA dataset, run:

$ python ./babi_story/train.py  

To train RMN on bAbI dialog dataset task 4, run:

$ python ./babi_dialog/train.py --task 4 --embedding concat --word_embed_dim 50

To use match, use_match flag is required:

$ python ./babi_dialog/train.py --task 4 --use_match True --embedding concat --word_embed_dim 50

To test on OOV dataset, is_oov flag is required:

$ python ./babi_dialog/train.py --task 4 --is_oov True --embedding concat --word_embed_dim 50