research-article

Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity Alignment

Authors:

Jianxin LiAuthors Info & Claims

WWW '23: Proceedings of the ACM Web Conference 2023

Pages 2499 - 2508

https://doi.org/10.1145/3543507.3583328

Published: 30 April 2023 Publication History

Abstract

The multi-modal entity alignment (MMEA) aims to find all equivalent entity pairs between multi-modal knowledge graphs (MMKGs). Rich attributes and neighboring entities are valuable for the alignment task, but existing works ignore contextual gap problems that the aligned entities have different numbers of attributes on specific modality when learning entity representations. In this paper, we propose a novel attribute-consistent knowledge graph representation learning framework for MMEA (ACK-MMEA) to compensate the contextual gaps through incorporating consistent alignment knowledge. Attribute-consistent KGs (ACKGs) are first constructed via multi-modal attribute uniformization with merge and generate operators so that each entity has one and only one uniform feature in each modality. The ACKGs are then fed into a relation-aware graph neural network with random dropouts, to obtain aggregated relation representations and robust entity representations. In order to evaluate the ACK-MMEA facilitated for entity alignment, we specially design a joint alignment loss for both entity and attribute evaluation. Extensive experiments conducted on two benchmark datasets show that our approach achieves excellent performance compared to its competitors.

References

[1]

Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held December 5-8, 2013, Lake Tahoe, Nevada, United States. 2787–2795. https://proceedings.neurips.cc/paper/2013/hash/1cecc7a77928ca8133fa24680a88d2f9-Abstract.html

Digital Library

[2]

Yixin Cao, Zhiyuan Liu, Chengjiang Li, Zhiyuan Liu, Juanzi Li, and Tat-Seng Chua. 2019. Multi-Channel Graph Neural Network for Entity Alignment. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 1452–1461. https://doi.org/10.18653/v1/p19-1140

[3]

Liyi Chen, Zhi Li, Yijun Wang, Tong Xu, Zhefeng Wang, and Enhong Chen. 2020. MMEA: Entity Alignment for Multi-modal Knowledge Graph. In Knowledge Science, Engineering and Management - 13th International Conference, KSEM 2020, Hangzhou, China, August 28-30, 2020, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 12274). Springer, 134–147. https://doi.org/10.1007/978-3-030-55130-8_12

Digital Library

[4]

Liyi Chen, Zhi Li, Tong Xu, Han Wu, Zhefeng Wang, Nicholas Jing Yuan, and Enhong Chen. 2022. Multi-modal Siamese Network for Entity Alignment. In KDD ’22: The 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, August 14 - 18, 2022. ACM, 118–126. https://doi.org/10.1145/3534678.3539244

Digital Library

[5]

Muhao Chen, Yingtao Tian, Kai-Wei Chang, Steven Skiena, and Carlo Zaniolo. 2018. Co-training Embeddings of Knowledge Graphs and Entity Descriptions for Cross-lingual Entity Alignment. In Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden. ijcai.org, 3998–4004. https://doi.org/10.24963/ijcai.2018/556

[6]

Muhao Chen, Yingtao Tian, Mohan Yang, and Carlo Zaniolo. 2017. Multilingual Knowledge Graph Embeddings for Cross-lingual Knowledge Alignment. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017. ijcai.org, 1511–1517. https://doi.org/10.24963/ijcai.2017/209

[7]

Hao Guo, Jiuyang Tang, Weixin Zeng, Xiang Zhao, and Li Liu. 2021. Multi-modal entity alignment in hyperbolic space. Neurocomputing 461 (2021), 598–607. https://doi.org/10.1016/j.neucom.2021.03.132

Digital Library

[8]

Fuzhen He, Zhixu Li, Qiang Yang, An Liu, Guanfeng Liu, Pengpeng Zhao, Lei Zhao, Min Zhang, and Zhigang Chen. 2019. Unsupervised Entity Alignment Using Attribute Triples and Relation Triples. In Database Systems for Advanced Applications - 24th International Conference, DASFAA 2019, Chiang Mai, Thailand, April 22-25, 2019, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 11446). Springer, 367–382. https://doi.org/10.1007/978-3-030-18576-3_22

Digital Library

[9]

Cheng Ji, Jianxin Li, Hao Peng, Jia Wu, Xingcheng Fu, Qingyun Sun, and Philip S. Yu. 2023. Unbiased and Efficient Self-Supervised Incremental Contrastive Learning. CoRR abs/2301.12104 (2023). https://doi.org/10.48550/arXiv.2301.12104 arXiv:2301.12104

[10]

Jin Jiang, Mohan Li, and Zhaoquan Gu. 2021. A Survey on Translating Embedding based Entity Alignment in Knowledge Graphs. In Sixth IEEE International Conference on Data Science in Cyberspace, DSC 2021, Shenzhen, China, October 9-11, 2021. IEEE, 187–194. https://doi.org/10.1109/DSC53577.2021.00033

[11]

Ke Liang, Lingyuan Meng, Meng Liu, Yue Liu, Wenxuan Tu, Siwei Wang, Sihang Zhou, Xinwang Liu, and Fuchun Sun. 2022. Reasoning over Different Types of Knowledge Graphs: Static, Temporal and Multi-Modal. CoRR abs/2212.05767 (2022). https://doi.org/10.48550/arXiv.2212.05767 arXiv:2212.05767

[12]

Zhenxi Lin, Ziheng Zhang, Meng Wang, Yinghui Shi, Xian Wu, and Yefeng Zheng. 2022. Multi-modal Contrastive Representation Learning for Entity Alignment. CoRR abs/2209.00891 (2022). https://doi.org/10.48550/arXiv.2209.00891 arXiv:2209.00891

[13]

Fangyu Liu, Muhao Chen, Dan Roth, and Nigel Collier. 2021. Visual Pivoting for (Unsupervised) Entity Alignment. In Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021, Virtual Event, February 2-9, 2021. AAAI Press, 4257–4266. https://ojs.aaai.org/index.php/AAAI/article/view/16550

[14]

Ye Liu, Hui Li, Alberto García-Durán, Mathias Niepert, Daniel Oñoro-Rubio, and David S. Rosenblum. 2019. MMKG: Multi-modal Knowledge Graphs. In The Semantic Web - 16th International Conference, ESWC 2019, Portorož, Slovenia, June 2-6, 2019, Proceedings(Lecture Notes in Computer Science, Vol. 11503). Springer, 459–474. https://doi.org/10.1007/978-3-030-21348-0_30

Digital Library

[15]

Zhiyuan Liu, Yixin Cao, Liangming Pan, Juanzi Li, and Tat-Seng Chua. 2020. Exploring and Evaluating Attributes, Values, and Structures for Entity Alignment. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020. Association for Computational Linguistics, 6355–6364. https://doi.org/10.18653/v1/2020.emnlp-main.515

[16]

Shichao Pei, Lu Yu, Robert Hoehndorf, and Xiangliang Zhang. 2019. Semi-Supervised Entity Alignment via Knowledge Graph Embedding with Awareness of Degree Difference. In The World Wide Web Conference, WWW 2019, San Francisco, CA, USA, May 13-17, 2019. ACM, 3130–3136. https://doi.org/10.1145/3308558.3313646

Digital Library

[17]

Xiaofei Shi and Yanghua Xiao. 2019. Modeling Multi-mapping Relations for Precise Cross-lingual Entity Alignment. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019. Association for Computational Linguistics, 813–822. https://doi.org/10.18653/v1/D19-1075

[18]

Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. http://arxiv.org/abs/1409.1556

[19]

Rui Sun, Xuezhi Cao, Yan Zhao, Junchen Wan, Kun Zhou, Fuzheng Zhang, Zhongyuan Wang, and Kai Zheng. 2020. Multi-modal Knowledge Graphs for Recommender Systems. In CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, 2020. ACM, 1405–1414. https://doi.org/10.1145/3340531.3411947

Digital Library

[20]

Zequn Sun, Chengming Wang, Wei Hu, Muhao Chen, Jian Dai, Wei Zhang, and Yuzhong Qu. 2020. Knowledge Graph Alignment Network with Gated Multi-Hop Neighborhood Aggregation. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7-12, 2020. 222–229. https://aaai.org/ojs/index.php/AAAI/article/view/5354

[21]

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA. 5998–6008. https://proceedings.neurips.cc/paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html

Digital Library

[22]

Zeshi Wang, Mohan Li, and Zhaoquan Gu. 2021. A Review of Entity Alignment based on Graph Convolutional Neural Network. In Sixth IEEE International Conference on Data Science in Cyberspace, DSC 2021, Shenzhen, China, October 9-11, 2021. IEEE, 144–151. https://doi.org/10.1109/DSC53577.2021.00027

[23]

Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. 2018. Cross-lingual Knowledge Graph Alignment via Graph Convolutional Networks. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018. 349–357. https://doi.org/10.18653/v1/d18-1032

[24]

Zhichun Wang, Qingsong Lv, Xiaohan Lan, and Yu Zhang. 2018. Cross-lingual knowledge graph alignment via graph convolutional networks. In Proceedings of the 2018 conference on empirical methods in natural language processing. 349–357.

[25]

Jiancan Wu, Xiang Wang, Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. 2021. Self-supervised Graph Learning for Recommendation. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021. ACM, 726–735. https://doi.org/10.1145/3404835.3462862

Digital Library

[26]

Yuting Wu, Xiao Liu, Yansong Feng, Zheng Wang, and Dongyan Zhao. 2020. Neighborhood Matching Network for Entity Alignment. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020. Association for Computational Linguistics, 6477–6487. https://doi.org/10.18653/v1/2020.acl-main.578

[27]

Fei Xia, Bin Li, Yixuan Weng, Shizhu He, Kang Liu, Bin Sun, Shutao Li, and Jun Zhao. 2022. LingYi: Medical Conversational Question Answering System based on Multi-modal Knowledge Graphs. CoRR abs/2204.09220 (2022). https://doi.org/10.48550/arXiv.2204.09220 arXiv:2204.09220

[28]

Guohai Xu, Hehong Chen, Feng-Lin Li, Fu Sun, Yunzhou Shi, Zhixiong Zeng, Wei Zhou, Zhongzhou Zhao, and Ji Zhang. 2021. AliMe MKG: A Multi-modal Knowledge Graph for Live-streaming E-commerce. In CIKM ’21: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, November 1 - 5, 2021. ACM, 4808–4812. https://doi.org/10.1145/3459637.3481983

Digital Library

[29]

Yuning You, Tianlong Chen, Yongduo Sui, Ting Chen, Zhangyang Wang, and Yang Shen. 2020. Graph Contrastive Learning with Augmentations. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. https://proceedings.neurips.cc/paper/2020/hash/3fe230348e9a12c13120749e3f9fa4cd-Abstract.html

[30]

Kaisheng Zeng, Chengjiang Li, Lei Hou, Juanzi Li, and Ling Feng. 2021. A comprehensive survey of entity alignment for knowledge graphs. AI Open 2 (2021), 1–13. https://doi.org/10.1016/j.aiopen.2021.02.002

[31]

Qingheng Zhang, Zequn Sun, Wei Hu, Muhao Chen, Lingbing Guo, and Yuzhong Qu. 2019. Multi-view Knowledge Graph Embedding for Entity Alignment. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019. ijcai.org, 5429–5435. https://doi.org/10.24963/ijcai.2019/754

[32]

Hao Zhu, Ruobing Xie, Zhiyuan Liu, and Maosong Sun. 2017. Iterative Entity Alignment via Joint Knowledge Embeddings. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, August 19-25, 2017. ijcai.org, 4258–4264. https://doi.org/10.24963/ijcai.2017/595

[33]

Xiangru Zhu, Zhixu Li, Xiaodan Wang, Xueyao Jiang, Penglei Sun, Xuwu Wang, Yanghua Xiao, and Nicholas Jing Yuan. 2022. Multi-Modal Knowledge Graph Construction and Application: A Survey. CoRR abs/2202.05786 (2022). arXiv:2202.05786https://arxiv.org/abs/2202.05786

Digital Library

Cited By

Zhao YZhang YSui XCai X(2025) ME A: A Multimodal Entity Entailment framework for multimodal Entity Alignment Information Processing & Management10.1016/j.ipm.2024.10395162:1(103951)Online publication date: Jan-2025
https://doi.org/10.1016/j.ipm.2024.103951
Su TSheng JWang SZhang XXu HLiu TCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity AlignmentProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680954(4436-4445)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680954
Zhang SZeng WTan ZZhao XXiao WSerra ESpezzano F(2024)M3: A Multi-Image Multi-Modal Entity Alignment DatasetProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679126(5415-5419)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679126
Show More Cited By

Index Terms

Attribute-Consistent Knowledge Graph Representation Learning for Multi-Modal Entity Alignment
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning
    2. Natural language processing
2. Theory of computation
  1. Semantics and reasoning

Recommendations

MMEA: Entity Alignment for Multi-modal Knowledge Graph
Knowledge Science, Engineering and Management
Abstract
Entity alignment plays an essential role in the knowledge graph (KG) integration. Though large efforts have been made on exploring the association of relational embeddings between different knowledge graphs, they may fail to effectively describe ...
Joint Entity Summary and Attribute Embeddings for Entity Alignment Between Knowledge Graphs
Hybrid Artificial Intelligent Systems
Abstract
Knowledge Graph (KG) is a popular way of storing facts about the real world entities, where nodes represent the entities and edges denote relations. KG is being used in many AI applications, so several large scale Knowledge Graphs (KGs) e.g., ...
Cross-lingual knowledge graph entity alignment based on relation awareness and attribute involvement
Abstract
Entity alignment is an effective means of matching entities from various knowledge graphs (KGs) that represent the equivalent real-world object. With the development of representation learning, recent entity alignment methods learn entity ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '23: Proceedings of the ACM Web Conference 2023

April 2023

4293 pages

ISBN:9781450394161

DOI:10.1145/3543507

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '23

Sponsor:

SIGWEB

WWW '23: The ACM Web Conference 2023

April 30 - May 4, 2023

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
438
Total Downloads

Downloads (Last 12 months)229
Downloads (Last 6 weeks)36

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhao YZhang YSui XCai X(2025) ME A: A Multimodal Entity Entailment framework for multimodal Entity Alignment Information Processing & Management10.1016/j.ipm.2024.10395162:1(103951)Online publication date: Jan-2025
https://doi.org/10.1016/j.ipm.2024.103951
Su TSheng JWang SZhang XXu HLiu TCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)IBMEA: Exploring Variational Information Bottleneck for Multi-modal Entity AlignmentProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680954(4436-4445)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680954
Zhang SZeng WTan ZZhao XXiao WSerra ESpezzano F(2024)M3: A Multi-Image Multi-Modal Entity Alignment DatasetProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679126(5415-5419)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679126
Wang YSun HWang JWang JTang WQi QSun SLiao J(2024)Towards Semantic Consistency: Dirichlet Energy Driven Robust Multi-Modal Entity Alignment2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00274(3559-3572)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00274
Yan ZPeng RWu H(2024)Similarity propagation based semi-supervised entity alignmentEngineering Applications of Artificial Intelligence10.1016/j.engappai.2023.107787130:COnline publication date: 2-Jul-2024
https://dl.acm.org/doi/10.1016/j.engappai.2023.107787
Fu TZhou G(2024)Temporal knowledge completion enhanced self-supervised entity alignmentJournal of Intelligent Information Systems10.1007/s10844-024-00878-5Online publication date: 13-Aug-2024
https://doi.org/10.1007/s10844-024-00878-5
Zhu BWang RWang JShao FWang K(2024)A survey: knowledge graph entity alignment research based on graph embeddingArtificial Intelligence Review10.1007/s10462-024-10866-457:9Online publication date: 3-Aug-2024
https://doi.org/10.1007/s10462-024-10866-4
Jin LChen J(2023)Self-supervised opinion summarization with multi-modal knowledge graphJournal of Intelligent Information Systems10.1007/s10844-023-00812-162:1(191-208)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1007/s10844-023-00812-1

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents