andreacossu / relation-network-pytorch

Implementation of Relation Network and Recurrent Relational Network using PyTorch v1.3. Original papers: (RN) https://arxiv.org/abs/1706.01427 (RRN): https://arxiv.org/abs/1711.08028

Hi there 👋

I am an assistant professor (RTD-A in Italy) at University of Pisa.
My research focuses on Continual Learning, with applications to Recurrent Neural Networks models and sequential data processing.

relation-network-pytorch's Issues

The accuracy is very low

I cloned the repo and finished the preliminary work (in Prerequisites), then simply used the following command to run the code:
python launch_rrn_babi.py --epochs=50000 --cuda

and here is part of the log:

......

Epoch  49000  /  50000
Train loss:  2.410700951099396 . Validation loss:  1.8214629411697387
Train accuracy:  0.16200000000000014 . Validation accuracy:  0.17250000000000024

Epoch  49500  /  50000
Train loss:  2.3995521495342254 . Validation loss:  1.8207665807008744
Train accuracy:  0.16660000000000014 . Validation accuracy:  0.17250000000000024

Epoch  50000  /  50000
Train loss:  2.405573818922043 . Validation loss:  1.8204271459579469
Train accuracy:  0.16240000000000024 . Validation accuracy:  0.16099999999999998

End training!
Testing...
Test accuracy:  0.15699999999999983
Test loss:  1.8213920128345489

What can I do to improve the accuracy?
Thanks in advance for any info you can provide!

Activation in RRN

Hi Andrea,
Thanks for this neat implementation!

I've noticed the MLP consists of linear layers followed by a tanh layer (code). However, in the RRN paper the authors mention using ReLu layers followed by a linear layer. Is this variation intentional?

Index Out of Range Error

I am trying to run the network using the tag --babi_tasks to execure only one task. I realised that to do so I have to download the full dataset and I did so. Nevertheless, I am getting this error:

Done reading babi!
Start training
Traceback (most recent call last):
File "D:\Documentos\Classroom_Activity_Recognition\Relation-Network-PyTorch\launch_rrn_babi.py", line 141, in
avg_train_losses, avg_train_accuracies, val_losses, val_accuracies = train(train_stories, validation_stories, args.epochs, lstm, rrn, criterion, optimizer, args.batch_size, args.no_save, device, result_folder)
File "D:\Documentos\Classroom_Activity_Recognition\Relation-Network-PyTorch\task\babi_task\rrn\train.py", line 48, in train
facts_emb, one_of_k, h_f = lstm.process_facts_rrn(facts_batch, h_f)
File "D:\Documentos\Classroom_Activity_Recognition\Relation-Network-PyTorch\src\models\LSTM.py", line 40, in process_facts_rrn
emb = self.embeddings(x) # (n_facts, n_words_facts, dim_emb)
File "C:\Users\alber\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\module.py", line 1071, in _call_impl
result = forward_call(*input, **kwargs)
File "C:\Users\alber\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\modules\sparse.py", line 158, in forward
return F.embedding(
File "C:\Users\alber\AppData\Local\Programs\Python\Python39\lib\site-packages\torch\nn\functional.py", line 2043, in embedding
return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self

is this an error in the code or am I doing something wrong?

Thanks!