daiquocnguyen / qgnn

Quaternion Graph Neural Networks (ACML 2021) (Pytorch and Tensorflow)

Quaternion Graph Neural Networks

This program provides the implementation of our QGNN as described in our paper, where we propose to learn embeddings for nodes and graphs within the Quaternion space.

QGNN	GNNs for knowledge graph completion

Usage

News

• September 2021: Our paper has been accepted to ACML 2021.
• August 2021: Release a Pytorch implementation of Gated Quaternion Graph Neural Networks.
• July 2021: Release a Pytorch implementation of Dual Quaternion Graph Neural Networks as described in our new paper about knowledge graph embeddings.
• June 2021: Release a Pytorch implementation of Simplifying Quaternion Graph Neural Networks.
• December 2020: Release a Pytorch implementation (v2) of QGNN for downstream tasks.
• November 2020: The extended abstract of our paper has been accepted to the NeurIPS 2020 Workshop on Differential Geometry meets Deep Learning (DiffGeo4DL).
• September 2020: A new blog on Quaternion Graph Neural Networks.

Requirements

• Python 3.7
• Networkx 2.3
• Scipy 1.3
• Tensorflow 1.14 or
• Pytorch 1.5.0 & CUDA 10.1

Training

Regarding knowledge graph completion:

SimQGNN$ python main_SimQGNN.py --dataset codex-s --num_iterations 4000 --eval_after 2000 --batch_size 1024 --lr 0.01 --emb_dim 128 --hidden_dim 128 --encoder QGNN

SimQGNN$ python main_SimQGNN.py --dataset codex-m --num_iterations 4000 --eval_after 2000 --batch_size 1024 --lr 0.005 --emb_dim 128 --hidden_dim 128 --encoder QGNN

SimQGNN$ python main_SimQGNN.py --dataset codex-l --num_iterations 2000 --eval_after 1000 --batch_size 1024 --lr 0.0001 --emb_dim 128 --hidden_dim 128 --encoder QGNN

Regarding node classification:

QGNN$ python train_node_cls.py --dataset cora --learning_rate 0.05 --hidden_size 16 --epochs 100 --fold 2

QGNN$ python train_node_cls.py --dataset citeseer --learning_rate 0.05 --hidden_size 16 --epochs 100 --fold 4

QGNN$ python train_node_cls.py --dataset pubmed --learning_rate 0.01 --hidden_size 64 --epochs 200 --fold 6

Regarding graph classification:

QGNN$ python train_graph_Sup.py --dataset IMDBBINARY --batch_size 4 --hidden_size 128 --fold_idx 2 --num_epochs 100 --num_GNN_layers 2 --learning_rate 0.0005 --model_name IMDBBINARY_bs4_hs128_fold2_k2_1

QGNN$ python train_graph_Sup.py --dataset DD --batch_size 4 --hidden_size 256 --fold_idx 5 --num_epochs 100 --num_GNN_layers 3 --learning_rate 0.0005 --model_name DD_bs4_hs256_fold5_k3_1

Cite

Please cite the paper whenever QGNN is used to produce published results or incorporated into other software:

@inproceedings{Nguyen2021QGNN,
	author={Dai Quoc Nguyen and Tu Dinh Nguyen and Dinh Phung},
	title={Quaternion Graph Neural Networks},
	booktitle={Asian Conference on Machine Learning},
	year={2021}
}

License

As a free open-source implementation, QGNN is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. All other warranties including, but not limited to, merchantability and fitness for purpose, whether express, implied, or arising by operation of law, course of dealing, or trade usage are hereby disclaimed. I believe that the programs compute what I claim they compute, but I do not guarantee this. The programs may be poorly and inconsistently documented and may contain undocumented components, features or modifications. I make no guarantee that these programs will be suitable for any application.

QGNN is licensed under the MIT License.

qgnn's Issues

Erroneously Low Accuracy in Tensorflow Version & RuntimeError in PyTorch Version

Hi there,

I've noticed that the Tensorflow version of the QGNN has erroneously low accuracy (python 3.8.2, tensorflow 2.4.0).

Script:
python train_node_cls.py --dataset cora --learning_rate 0.05 --hidden_size 16 --epochs 100 --fold 2

PyTorch:

Epoch: 0100 loss_train: 0.3678 acc_train: 0.8960 loss_val: 0.4513 acc_val: 0.8606 time: 0.3034s
Optimization Finished!
Total time elapsed: 33.9790s
Test set results: loss= 0.3706 accuracy= 0.8934

Tensorflow:

Epoch: 0100 train_loss= 0.50750 train_acc= 0.90352 val_loss= nan val_acc= 0.14196 time= 0.03599
Optimization Finished!
Test set results at final epoch: cost= nan accuracy= 0.13481 time= 0.02022

As you can see cost is a NaN and accuracy is 0.13481, which is far lower than that of the PyTorch version (0.8934).

Also, I get the following error when I try to run the script below using the PyTorch version (torch 1.6.0).

Script:
python train_node_cls.py --dataset citeseer --learning_rate 0.05 --hidden_size 16 --epochs 100 --fold 4

PyTorch:

train_node_cls.py:56: RuntimeWarning: divide by zero encountered in power
  r_inv = np.power(rowsum, -1).flatten()
Traceback (most recent call last):
  File "train_node_cls.py", line 129, in <module>
    train(epoch)
  File "train_node_cls.py", line 97, in train
    output = model(features, adj)
  File "/home/satyoshi/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "train_node_cls.py", line 84, in forward
    x = self.q4gnn1(x, adj)
  File "/home/satyoshi/.local/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/mnt/c/data/programming/machinelearning/QGNN-master/QGNN_pytorch/q4gnn.py", line 77, in forward
    support = torch.mm(x, hamilton)  # Hamilton product, quaternion multiplication!
RuntimeError: Expected object of scalar type Double but got scalar type Float for argument #3 'mat2' in call to _th_addmm_out

Tensorflow:

Epoch: 0100 train_loss= 0.61921 train_acc= 0.85939 val_loss= nan val_acc= 0.06333 time= 0.06901
Optimization Finished!
Test set results at final epoch: cost= nan accuracy= 0.06132 time= 0.04012

The error does not occur with the Tensorflow version, but still it has the same NaN issue as above.

Best,
JS

Low accuracy

Hi:
Can you provide specific hyper-parameters for some datasets used in paper, such as MUTAG, PROTEIN. I use default parameter to train these datasets, I found accuracy is very low which is far less than the result in paper.