liyichen-cly / mmea

MMEA: Entity Alignment for Multi-Modal Knowledge Graphs, KSEM 2020

MMEA: Entity Alignment for Multi-Modal Knowledge Graphs

Model code and datasets for paper "MMEA: Entity Alignment for Multi-Modal Knowledge Graphs" published in Proceedings of the 13th International Conference on Knowledge Science, Engineering and Management (KSEM'2020).

Entity alignment plays an essential role in the knowledge graph (KG) integration. Though large efforts have been made on exploring the association of relational embeddings between different knowledge graphs, they may fail to effectively describe and integrate the multimodal knowledge in the real application scenario. To that end, in this paper, we propose a novel solution called Multi-Modal Entity Alignment (MMEA) to address the problem of entity alignment in a multi-modal view. Specifically, we first design a novel multi-modal knowledge embedding method to generate the entity representations of relational, visual and numerical knowledge, respectively. Along this line, multiple representations of different types of knowledge will be integrated via a multimodal knowledge fusion module. Extensive experiments on two public datasets clearly demonstrate the effectiveness of the MMEA model with a significant margin compared with the state-of-the-art methods.

Dataset

Three public multi-modal knowledge graphs with relational, numerical and visual knowledge from paper "MMKG: Multi-Modal Knowledge Graphs", i.e., FB15k, DB15k and YAGO15k. There are sameAs links between FB15k and DB15k as well as between FB15k and YAGO15k, which could be regarded as alignment relations. Please click here to download the datasets.

Code

Our code was implemented by extending the public benchmark OpenEA, therefore we only public the model code to avoid repetition. We appreciate the authors for making OpenEA open-sourced.

Dependencies

• Python 3.6
• Tensorflow 1.10
• Numpy 1.16

Citation

If you use this model or code, please kindly cite it as follows:

@inproceedings{chen2020mmea,
  title={MMEA: Entity Alignment for Multi-modal Knowledge Graph},
  author={Chen, Liyi and Li, Zhi and Wang, Yijun and Xu, Tong and Wang, Zhefeng and Chen, Enhong},
  booktitle={International Conference on Knowledge Science, Engineering and Management},
  pages={134--147},
  year={2020},
  organization={Springer}
}

---

Last but not least, if you have any difficulty or question in implementations, please send your email to [email protected].

mmea's Issues

some question

I see no information in the model code about how to fuse the embedding and how to apply numerical knowledge to the model, can you please share

Would you please share the running process of code?

Hi Liyi, i want to re-run your MMEA code, but i don't how to use Multi-modal datasets in your code and how to run your code? I would be very grateful If you can share your running script and detailed instructions for how to use Multi-modal datasets? Would you please give me some guidance？

I will be looking forward to your reply，thanks a lot.

Request for args.json file.

Hi, your code is implemented based on OpenEA framework. So, could you please provide the args.json file containing the hyperparameters.