research-article

Open access

A Neighborhood-Attention Fine-grained Entity Typing for Knowledge Graph Completion

Authors:

Weisi HanAuthors Info & Claims

WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

Pages 1525 - 1533

https://doi.org/10.1145/3488560.3498395

Published: 15 February 2022 Publication History

Abstract

Knowledge graph (KG) entity typing focuses on inferring possible entity type instances, which is a significant subtask of knowledge graph completion (KGC). Existing entity typing methods usually exploit the entity representation to model the transmission between entities and their types, which cannot fully explore the fine-grained entity typing on identifying the semantic type of an entity. To address these issues, we propose Neighborhood-Attention Neural Fine-Grained Entity Typing (AttEt), which considers the neighborhood information of the entities from KGs to bridge entities and their types together. In this paper, AttEt first develops a type-specific attention mechanism to aggregate the neighborhood knowledge of the given entity with type-specific weights. These weights are beneficial to capture various characteristics for different types of the entity, and further imply the complex correlation among these fine-grained types. Then, AttEt adaptively integrates the aggregated neighbor-level representation with entity inherent embedding to calculate the matching score between the entity and its candidate type. Besides, many entities are sparse in their relations with other entities in KGs, which makes the entity typing task more challenging. To solve this problem, we present a smooth strategy on relation-sparsity entities to improve the robustness of the model. Extensive experiments on two real-world datasets (Freebase and YAGO) show that AttEt significantly outperforms state-of-the-art baselines in the HITS@1 by 2.11% on Freebase and by 8.42% on YAGO, respectively.

Supplementary Material

MP4 File (WSDM22-fp171.mp4)

Knowledge graph (KG) entity typing focuses on inferring possible entity type instances, which is a significant subtask of knowledge graph completion (KGC). Existing entity typing methods usually exploit the entity representation to model the transmission between entities and their types, which cannot fully explore the fine-grained entity typing on identifying the semantic type of an entity. To address these issues, we propose Neighborhood-Attention Neural Fine-Grained Entity Typing (AttEt), which considers the neighborhood information of the entities from KGs to bridge entities and their types together. Besides, many entities are sparse in their relations with other entities in KGs, which makes the entity typing task more challenging. To solve this problem, we present a smooth strategy on relation-sparsity entities to improve the robustness of the model. Extensive experiments on two real-world datasets (Freebase and YAGO) show that AttEt significantly outperforms state-of-the-art baselines in the HITS@1 by 2.11% on Freebase and by 8.42% on YAGO, respectively.

Download
44.45 MB

References

[1]

Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008a. Freebase:a collaboratively created graph database for structuring human knowledge, In In Proceedings of KDD. In processings of the 2008 ACM SIGMOD international Conference on Management of Data, 1247--1250.

[2]

Kurt D. Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008b. Freebase: a collaboratively created graph database for structuring human knowledge. In Sigmod Conference .

Digital Library

[3]

A. Bordes, N. Usunier, and A. Garcia-Dur!'"aan. 2013. Translating Embeddings for Modeling Multi-relational Data. In In Proceedings of NIPS. 2787--2795.

[4]

Tongfei Chen, Yunmo Chen, and Benjamin Van Durme. 2020. Hierarchical Entity Typing via Multi-level Learning to Rank. In Proceedings of ACL . Association for Computational Linguistics, Online, 8465--8475.

[5]

Eunsol Choi, Omer Levy, Yejin Choi, and Luke Zettlemoyer. 2018. Ultra-Fine Entity Typing. In Proceedings of ACL . Association for Computational Linguistics, Melbourne, Australia, 87--96.

[6]

Wanyun Cui, Yanghua Xiao, Haixun Wang, Yangqiu Song, Seung-won Hwang, and Wei Wang. 2017. KBQA: learning question answering over QA corpora and knowledge bases. Proceedings of the VLDB Endowment, Vol. 10, 5 (Jan. 2017), 565--576. https://doi.org/10.14778/3055540.3055549

Digital Library

[7]

John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, Vol. 12, 7 (2011).

Digital Library

[8]

Hannaneh Hajishirzi, Leila Zilles, Daniel S. Weld, and Luke Zettlemoyer. 2013. Joint Coreference Resolution and Named-Entity Linking with Multi-Pass Sieves. In Proceedings of EMNLP . Association for Computational Linguistics, Seattle, Washington, USA, 289--299.

[9]

Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. 2019. Knowledge Graph Embedding Based Question Answering. In Proceedings of WSDM (WSDM '19). Association for Computing Machinery, New York, NY, USA, 105--113.

Digital Library

[10]

Prachi Jain, Pankaj Kumar, Soumen Chakrabarti, et al. 2018. Type-sensitive knowledge base inference without explicit type supervision. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). 75--80.

[11]

G. Ji, S. He, L. Xu, K. Liu, and J. Zhao. 2015. Knowledge graph embedding via dynamic mapping matrix. In In Proceedings of ACL . 687--696.

[12]

Hailong Jin, Lei Hou, Juanzi Li, and Tiansi Dong. 2018. Attributed and Predictive Entity Embedding for Fine-Grained Entity Typing in Knowledge Bases. In Proceedings of ACL, Emily M. Bender, Leon Derczynski, and Pierre Isabelle (Eds.). Association for Computational Linguistics, 282--292.

[13]

Hailong Jin, Lei Hou, Juanzi Li, and Tiansi Dong. 2019. Fine-Grained Entity Typing via Hierarchical Multi Graph Convolutional Networks. In Proceedings of EMNLP, Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 4968--4977.

[14]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In Proceedings of ICLR . OpenReview.net.

[15]

J. Lehmann, R. Isele, and M. Jakob. 2015. DBpedia: A large-scale, multilingual knowledge base extracted from wikipedia. Semantic Web, Vol. 6, 2 (2015), 167--195.

[16]

Jian Li, Yong Liu, Rong Yin, Hua Zhang, Lizhong Ding, and Weiping Wang. 2018. Multi-Class Learning: From Theory to Algorithm. NeurIPS, Vol. 31 (2018), 1593--1602.

[17]

Yankai Lin, Zhiyuan Liu, Xuan Zhu, Xuan Zhu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of AAAI. 2181--2187.

Digital Library

[18]

Shuman Liu, Hongshen Chen, Zhaochun Ren, Yang Feng, Qun Liu, and Dawei Yin. 2018. Knowledge Diffusion for Neural Dialogue Generation. In Proceedings of ACL . Association for Computational Linguistics, Melbourne, Australia, 1489--1498.

[19]

Yong Liu, Shizhong Liao, Shali Jiang, Lizhong Ding, Hailun Lin, and Weiping Wang. 2020. Fast Cross-Validation for Kernel-Based Algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, 5 (2020), 1083--1096. https://doi.org/10.1109/TPAMI.2019.2892371

[20]

Changsung Moon, Paul Jones, and Nagiza F Samatova. 2017. Learning entity type embeddings for knowledge graph completion. In Proceedings of CIKM . 2215--2218.

Digital Library

[21]

Dai Quoc Nguyen, Thanh Vu, Tu Dinh Nguyen, Dat Quoc Nguyen, and Dinh Phung. 2019. A Capsule Network-based Embedding Model for Knowledge Graph Completion and Search Personalization. In Proceedings of NAACL . Association for Computational Linguistics, Minneapolis, Minnesota, 2180--2189.

[22]

Maximilian Nickel, Lorenzo Rosasco, and Tomaso Poggio. 2016. Holographic Embeddings of Knowledge Graphs. In Proceedings of AAAI (AAAI'16). AAAI Press, 1955--1961.

[23]

Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011a. A three-way model for collective learning on multi-relational data. In Proceedings of ICML, Vol. 11. 809--816.

[24]

Maximilian Nickel, Volker Tresp, and Hans Peter Kriegel. 2011b. A Three-Way Model for Collective Learning on Multi-Relational Data. In International Conference on Machine Learning, ICML 2011. 809--816.

[25]

Yasumasa Onoe, Michael Boratko, and Greg Durrett. 2021. Modeling Fine-Grained Entity Types with Box Embeddings. CoRR, Vol. abs/2101.00345 (2021).

[26]

Patrick Pantel, Thomas Lin, and Michael Gamon. 2012. Mining Entity Types from Query Logs via User Intent Modeling. In Proceedings of ACL. Association for Computational Linguistics, Jeju Island, Korea, 563--571.

[27]

Heiko Paulheim and Christian Bizer. 2013. Type Inference on Noisy RDF Data. In Proceedings of the 12th International Semantic Web Conference - Part I (ISWC '13). Springer-Verlag, 510--525. https://doi.org/10.1007/978--3--642--41335--3_32

[28]

Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2020. A Survey on Knowledge Graph-Based Recommender Systems . arXiv:2003.00911 [cs, stat] (Feb. 2020). http://arxiv.org/abs/2003.00911 arXiv: 2003.00911.

[29]

Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum. 2007. Yago: a core of semantic knowledge. In Proceedings of WWW. 697--706.

Digital Library

[30]

Thé o Trouillon and Maximilian Nickel. 2017. Complex and Holographic Embeddings of Knowledge Graphs: A Comparison. CoRR, Vol. abs/1707.01475 (2017).

[31]

Thé o Trouillon, Johannes Welbl, Sebastian Riedel, É ric Gaussier, and Guillaume Bouchard. 2016. Complex Embeddings for Simple Link Prediction. In Proceedings of ICML. 2071--2080.

[32]

Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2017. Graph Attention Networks. CoRR, Vol. abs/1710.10903 (2017).

[33]

Hongwei Wang, Hongyu Ren, and Jure Leskovec. 2020. Entity Context and Relational Paths for Knowledge Graph Completion. CoRR, Vol. abs/2002.06757 (2020). arxiv: 2002.06757

[34]

Xiang Wang, Dingxian Wang, Canran Xu, Xiangnan He, Yixin Cao, and Tat-Seng Chua. 2018. Explainable Reasoning over Knowledge Graphs for Recommendation . arXiv:1811.04540 [cs] (Nov. 2018). http://arxiv.org/abs/1811.04540 arXiv: 1811.04540.

[35]

Z. Wang, J. Zhang, J. Feng, and Z. Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In In Proceedings of AAAI . 1112--1119.

[36]

H. Xiao, M. Huang, and X. Zhu. 2016. TransG: A Generative Model for Knowledge Graph Embedding. In Proceedings of ACL . 2316--2325.

[37]

Ruobing Xie, Zhiyuan Liu, Jia Jia, Huanbo Luan, and Maosong Sun. 2016. Representation Learning of Knowledge Graphs with Entity Descriptions. In Proceedings of AAAI . AAAI Press, 2659--2665.

[38]

Ji Xin, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2018. Improving Neural Fine-Grained Entity Typing With Knowledge Attention. In Proceedings of AAAI, Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 5997--6004.

[39]

Bo Xu, Yi Zhang, Jiaqing Liang, Yanghua Xiao, Seung-won Hwang, and Wei Wang. 2016. Cross-Lingual Type Inference. In Proceedings of DASFAA (Lecture Notes in Computer Science), Vol. 9642. Springer, 447--462.

[40]

Peng Xu and Denilson Barbosa. 2018. Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss. In Proceedings of ACL . Association for Computational Linguistics, New Orleans, Louisiana, 16--25.

[41]

Yadollah Yaghoobzadeh, Heike Adel, and Hinrich Schü tze. 2018. Corpus-Level Fine-Grained Entity Typing. J. Artif. Intell. Res., Vol. 61 (2018), 835--862.

[42]

Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2014. Embedding Entities and Relations for Learning and Inference in Knowledge Bases. CoRR, Vol. abs/1412.6575 (2014).

[43]

Limin Yao, Sebastian Riedel, and Andrew McCallum. 2013. Universal schema for entity type prediction. In Proceedings of workshop on Automated knowledge base construction (AKBC '13). Association for Computing Machinery, New York, NY, USA, 79--84.

Digital Library

[44]

Dani Yogatama, Daniel Gillick, and Nevena Lazic. 2015. Embedding Methods for Fine Grained Entity Type Classification. In Proceedings of ACL . Association for Computational Linguistics, Beijing, China, 291--296.

[45]

Chuxu Zhang, Huaxiu Yao, Chao Huang, Meng Jiang, Zhenhui Li, and Nitesh V. Chawla. 2020. Few-Shot Knowledge Graph Completion . Proceedings of AAAI Conference on Artificial Intelligence, Vol. 34, 03 (April 2020), 3041--3048. Number: 03.

[46]

Richong Zhang, Fanshuang Kong, Chenyue Wang, and Yongyi Mao. 2018. Embedding of Hierarchically Typed Knowledge Bases. In Proceedings of AAAI, Sheila A. McIlraith and Kilian Q. Weinberger (Eds.). AAAI Press, 2046--2053.

[47]

Yu Zhao, Anxiang Zhang, Ruobing Xie, Kang Liu, and Xiaojie Wang. 2020. Connecting Embeddings for Knowledge Graph Entity Typing. In Proceedings of ACL . Association for Computational Linguistics, Online.

[48]

Ben Zhou, Daniel Khashabi, Chen-Tse Tsai, and Dan Roth. 2018. Zero-Shot Open Entity Typing as Type-Compatible Grounding. In Proceedings of ACL. Association for Computational Linguistics, Brussels, Belgium, 2065--2076.

Cited By

Xu JYu JCai YChua T(2024)Dual Contrastive Learning for Cross-Domain Named Entity RecognitionACM Transactions on Information Systems10.1145/367887942:6(1-33)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3678879
Wang YGe XWang BKuo CBaeza-Yates RBonchi F(2024)AsyncET: Asynchronous Representation Learning for Knowledge Graph Entity TypingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671832(3267-3276)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671832
Miao LWu YYang Q(2024)Instance type completion in equipment knowledge graph based on translation modelInternational Journal of Modeling, Simulation, and Scientific Computing10.1142/S1793962324500351Online publication date: 23-Jul-2024
https://doi.org/10.1142/S1793962324500351
Show More Cited By

Index Terms

A Neighborhood-Attention Fine-grained Entity Typing for Knowledge Graph Completion
1. Computing methodologies
  1. Artificial intelligence
    1. Knowledge representation and reasoning

Recommendations

Learning Entity Type Embeddings for Knowledge Graph Completion
CIKM '17: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management

Missing data is a severe problem for algorithms that operate over knowledge graphs (KGs). Most previous research in KG completion has focused on the problem of inferring missing entities and missing relation types between entities. However, in addition ...
Fine-Grained Entity Typing for Relation-Sparsity Entities
Database Systems for Advanced Applications
Abstract
This paper works on fine-grained entity typing without using external knowledge for Knowledge Graphs (KGs). Aiming at identifying the semantic type of an entity, this task has been studied predominantly in KGs. Provided with dense enough relations ...
It’s All in the Name: Entity Typing Using Multilingual Language Models
The Semantic Web: ESWC 2022 Satellite Events
Abstract
The entity type information in Knowledge Graphs (KGs) of different languages plays an important role in a wide range of Natural Language Processing applications. However, the entity types in KGs are often incomplete. Multilingual entity typing is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining

February 2022

1690 pages

ISBN:9781450391320

DOI:10.1145/3488560

General Chairs:
K. Selcuk Candan
Arizona State University, USA
,
Huan Liu
Arizona State University, USA
,
Program Chairs:
Leman Akoglu
Carnegie Mellon University, USA
,
Xin Luna Dong
Meta Platforms, Inc. (former Facebook), USA
,
Jiliang Tang
Michigan State University, USA

Copyright © 2022 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 February 2022

Check for updates

Author Tags

Qualifiers

Research-article

Conference

WSDM '22

Sponsor:

WSDM '22: The Fifteenth ACM International Conference on Web Search and Data Mining

February 21 - 25, 2022

AZ, Virtual Event, USA

Acceptance Rates

Overall Acceptance Rate 498 of 2,863 submissions, 17%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

16
Total Citations
View Citations
1,196
Total Downloads

Downloads (Last 12 months)243
Downloads (Last 6 weeks)23

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu JYu JCai YChua T(2024)Dual Contrastive Learning for Cross-Domain Named Entity RecognitionACM Transactions on Information Systems10.1145/367887942:6(1-33)Online publication date: 18-Oct-2024
https://dl.acm.org/doi/10.1145/3678879
Wang YGe XWang BKuo CBaeza-Yates RBonchi F(2024)AsyncET: Asynchronous Representation Learning for Knowledge Graph Entity TypingProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671832(3267-3276)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671832
Miao LWu YYang Q(2024)Instance type completion in equipment knowledge graph based on translation modelInternational Journal of Modeling, Simulation, and Scientific Computing10.1142/S1793962324500351Online publication date: 23-Jul-2024
https://doi.org/10.1142/S1793962324500351
Lu YYang DWang PRosso PCudre-Mauroux P(2024)Schema-Aware Hyper-Relational Knowledge Graph Embeddings for Link PredictionIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2023.332349936:6(2614-2628)Online publication date: Jun-2024
https://doi.org/10.1109/TKDE.2023.3323499
Wang ZZhang WDai J(2024)An Information Completion Method Based on Security Knowledge Graph with Fusing Neighborhood Information2024 2nd International Conference on Signal Processing and Intelligent Computing (SPIC)10.1109/SPIC62469.2024.10691469(303-307)Online publication date: 20-Sep-2024
https://doi.org/10.1109/SPIC62469.2024.10691469
Xie ZZhang GFang Z(2024)Power Line Detection Based on Subpixel-Neighborhood Attention in Complex Terrain BackgroundsIEEE Sensors Journal10.1109/JSEN.2023.334787724:9(14493-14502)Online publication date: 1-May-2024
https://doi.org/10.1109/JSEN.2023.3347877
Guo YMao CYue DLeng T(2024)Enhancing Commonsense Reasoning through Entity Type Knowledge Graph Completion2024 5th International Conference on Artificial Intelligence and Electromechanical Automation (AIEA)10.1109/AIEA62095.2024.10692845(298-302)Online publication date: 14-Jun-2024
https://doi.org/10.1109/AIEA62095.2024.10692845
Gan YSu ZLu GZhang PCui AJiang JChen D(2024)Entity type inference based on path walking and inter-types relationshipsData & Knowledge Engineering10.1016/j.datak.2024.102337153(102337)Online publication date: Sep-2024
https://doi.org/10.1016/j.datak.2024.102337
Liu LHuang WShu JZhao H(2024)Knowledge graph relation prediction based on graph transformationApplied Intelligence10.1007/s10489-024-06080-y55:3Online publication date: 30-Dec-2024
https://doi.org/10.1007/s10489-024-06080-y
Cao YXu KCheng YYan JHe Z(2024)HENF: Hierarchical Entity Neighbor Multi-Relational Fusion Network for Knowledge Graph CompletionData Science10.1007/978-981-97-8746-3_17(255-268)Online publication date: 31-Oct-2024
https://doi.org/10.1007/978-981-97-8746-3_17
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents