research-article

Tucker decomposition-based temporal knowledge graph completion

Authors:

Tong LiuAuthors Info & Claims

Volume 238, Issue C

https://doi.org/10.1016/j.knosys.2021.107841

Published: 28 February 2022 Publication History

Abstract

Knowledge graphs have been demonstrated to be an effective tool for numerous intelligent applications. However, a large amount of valuable knowledge still exists implicitly in the knowledge graphs. To enrich the existing knowledge graphs, recent years have witnessed that many algorithms for link prediction and knowledge graphs embedding have been designed to infer new facts. But most of these studies focus on the static knowledge graphs and ignore the temporal information which reflects the validity of knowledge. Developing the model for temporal knowledge graphs completion is an increasingly important task. In this paper, we build a new tensor decomposition model for temporal knowledge graphs completion inspired by the Tucker decomposition of order-4 tensor. Furthermore, to further improve the basic model performance, we provide three kinds of methods including cosine similarity, contrastive learning, and reconstruction-based to incorporate the prior knowledge into the proposed model. Because the core tensor contains a large number of parameters on the proposed model, thus we present two embedding regularization schemes to avoid the over-fitting problem. By combining these two kinds of regularization with the proposed model, our model outperforms baselines with an explicit margin on three temporal datasets (i.e. ICEWS2014, ICEWS05-15, GDELT).

References

[1]

Koren Y., Bell R., Volinsky C., Matrix factorization techniques for recommender systems, Proc. Comput. (2009) 30–37.

[2]

L. Dong, F. Wei, M. Zhou, K. Xu, Question answering over freebase with multi-column convolutional neural networks, in: Proceedings of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, 2015, pp. 260–269.

[3]

C. Xiong, J.P. Callan, Query expansion with freebase, in: Proceedings of the International Conference on the Theory of Information Retrieval, 2015, pp. 111—120.

[4]

Z. Wang, C. Chen, W. Li, Predictive network representation learning for link prediction, in: Proceedings of International ACM SIGIR Conference on Research and Development in Information Retrieval, 2017, pp. 969—972.

[5]

T. Lacroix, N. Usunier, G. Obozinski, Canonical tensor decomposition for knowledge base completion, in: Proceedings of the International Conference on Machine Learning, 2018.

[6]

I. Balažević, C. Allen, T.M. Hospedales, TuckER: Tensor factorization for knowledge graph completion, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, 2019, pp. 5185–5194.

[7]

T. Lacroix, G. Obozinski, N. Usunier, Tensor decompositions for temporal knowledge base completion, in: Proceedings of International Conference on Learning Representations, 2020.

[8]

Håstad J., Tensor rank is NP-complete, J. Algorithms (1990) 644–654.

[9]

P. Jain, S. Rathi, . Mausam, S. Chakrabarti, Temporal knowledge base completion: New algorithms and evaluation protocols, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2020, pp. 3733–3747.

[10]

B. Yang, S.W.-t. Yih, X. He, J. Gao, L. Deng, Embedding entities and relations for learning and inference in knowledge bases, in: Proceedings of the International Conference on Learning Representations, 2015.

[11]

T. Trouillon, J. Welbl, S. Riedel, E. Gaussier, G. Bouchard, Complex embeddings for simple link prediction, in: Proceedings of the International Conference on Machine Learning, 2016, pp. 2071—2080.

[12]

S.M. Kazemi, D. Poole, SimplE embedding for link prediction in knowledge graphs, in: Proceedings of the International Conference on Neural Information Processing Systems, 2018, pp. 4289—4300.

[13]

R. Goel, S.M. Kazemi, M. Brubaker, P. Poupart, Diachronic embedding for temporal knowledge graph completion, in: Proceedings of AAAI Conference on Artificial Intelligence, 2020, pp. 3988–3995.

[14]

A. Bordes, N. Usunier, A. Garcia-Durán, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational data, in: Proceedings of the International Conference on Neural Information Processing Systems, 2013, pp. 2787—2795.

[15]

G. Ji, S. He, L. Xu, K. Liu, J. Zhao, Knowledge graph embedding via dynamic mapping matrix, in: Proceedings of Annual Meeting of the Association for Computational Linguistics, 2015.

[16]

T. Dettmers, M. Pasquale, S. Pontus, S. Riedel, Convolutional 2D knowledge graph embeddings, in: Proceedings of AAAI Conference on Artificial Intelligence, 2018, pp. 1811–1818.

[17]

M. Schlichtkrull, T.N. Kipf, P. Bloem, R.v.d. Berg, I. Titov, M. Welling, Modeling relational data with graph convolutional networks, in: Proc. Semant. Web, 2017, pp. 593–607.

[18]

M. Nickel, V. Tresp, H.-P. Kriegel, A three-way model for collective learning on multi-relational data, in: Proceedings of the International Conference on Machine Learning, 2011, pp. 809—816.

[19]

M. Nickel, L. Rosasco, T. Poggio, Holographic embeddings of knowledge graphs, in: Proceedings of AAAI Conference on Artificial Intelligence, 2016, pp. 1955—1961.

[20]

T. Jiang, T. Liu, T. Ge, L. Sha, S. Li, B. Chang, Z. Sui, Encoding temporal information for time-aware link prediction, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2016, pp. 2350–2354.

[21]

S.S. Dasgupta, S.N. Ray, P. Talukdar, HyTE: Hyperplane-based temporally aware knowledge graph embedding, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018, pp. 2001–2011.

[22]

Z. Wang, J. Zhang, J. Feng, Z. Chen, Knowledge graph embedding by translating on hyperplanes, in: Proceedings of AAAI Conference on Artificial Intelligence, 2014, pp. 1112—1119.

[23]

A. García-Durán, S. Dumancic, M. Niepert, Learning sequence encoders for temporal knowledge graph completion, in: Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2018, pp. 4816—4821.

[24]

Ma Y., Tresp V., Daxberger E., Embedding models for episodic knowledge graphs, Proc. J. Web Semant. (2019).

[25]

C. Xu, M. Nayyeri, F. Alkhoury, H. Yazdi, J. Lehmann, Temporal knowledge graph completion based on time series gaussian embedding, in: Proceedings of the Semantic Web, 2020, pp. 654–671.

[26]

C. Xu, Y.-Y. Chen, M. Nayyeri, J. Lehmann, Temporal knowledge graph completion using a linear temporal regularizer and multivector embeddings, in: Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 2569–2578.

[27]

Y. Xu, H. E, M. Song, W. Song, X. Lv, H. Wang, J. Yang, RTFE: A recursive temporal fact embedding framework for temporal knowledge graph completion, in: NAACL-HLT, 2021.

[28]

Hitchcock F.L., The expression of a tensor or a polyadic as a sum of products, Proc. Stud. Appl. Math. (1927) 164–189.

[29]

L.R. Tucker, The extension of factor analysis to three-dimensional matrices, in: Proceedings of Contributions to Mathematical Psychology, 1964, pp. 110–127.

[30]

Kolda T., Bader B., Tensor decompositions and applications, 2009, pp. 455–-500.

[31]

Boschee E., Lautenschlager J., O’Brien S., Shellman S., Starz J., Ward M., ICEWS coded event data, 2015.

[32]

K. Leetaru, P.A. Schrodt, GDELT: Global data on events, location, and tone, in: Proceedings of ISA Annual Convention.

[33]

R. Trivedi, H. Dai, Y. Wang, L. Song, Know-Evolve: Deep temporal reasoning for dynamic knowledge graphs, in: Proceedings of the International Conference on Machine Learning, 2017, pp. 3462—3471.

[34]

Nickel M., Murphy K., Tresp V., Gabrilovich E., A review of relational machine learning for knowledge graphs, 2016, pp. 11–33.

[35]

A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. Devito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, Automatic differentiation in PyTorch, in: Proceedings of the International Conference on Neural Information Processing Systems, 2017.

[36]

J.C. Duchi, E. Hazan, Y. Singer, Adaptive subgradient methods for online learning and stochastic optimization, in: Proceedings of Machine Learning Research, 2011, pp. 2121—2159.

Cited By

Yu JLi ZMa GWang JZou TZhou G(2025)Low-rank sparse fully-connected tensor network for tensor completionPattern Recognition10.1016/j.patcog.2024.111000158:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.patcog.2024.111000
Fan WPang KLu PTian C(2024)Making It Tractable to Detect and Correct Errors in GraphsACM Transactions on Database Systems10.1145/370231549:4(1-75)Online publication date: 2-Nov-2024
https://dl.acm.org/doi/10.1145/3702315
Wang JCui ZWang BPan SGao JYin BGao WChua TNgo CKa-Wei Lee RKumar RLauw H(2024)IME: Integrating Multi-curvature Shared and Specific Embedding for Temporal Knowledge Graph CompletionProceedings of the ACM Web Conference 202410.1145/3589334.3645361(1954-1962)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645361
Show More Cited By

Index Terms

Tucker decomposition-based temporal knowledge graph completion

Index terms have been assigned to the content through auto-classification.

Recommendations

A low-rank and sparse enhanced Tucker decomposition approach for tensor completion
Abstract
In this paper, we introduce a unified low-rank and sparse enhanced Tucker decomposition model for tensor completion. Our model possesses a sparse regularization term to promote a sparse core of the Tucker decomposition, which is beneficial for ...
Low-Rank Tensor Tucker Decomposition for Hyperspectral Images Super-Resolution
Intelligent Robotics and Applications
Abstract
Super-resolution is an important way to improve the spatial resolution of Hyperspectral images (HSIs). In this paper, we propose a super-resolution method based on low-rank tensor Tucker Decomposition and weighted 3D total variation (TV) for HSIs. ...
Static and Streaming Tucker Decomposition for Dense Tensors
Given a dense tensor, how can we efficiently discover hidden relations and patterns in static and online streaming settings? Tucker decomposition is a fundamental tool to analyze multidimensional arrays in the form of tensors. However, existing Tucker ...

Comments

Information & Contributors

Information

Published In

cover image Knowledge-Based Systems

Knowledge-Based Systems Volume 238, Issue C

Feb 2022

536 pages

ISSN:0950-7051

Issue’s Table of Contents

Copyright © 2021.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 28 February 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu JLi ZMa GWang JZou TZhou G(2025)Low-rank sparse fully-connected tensor network for tensor completionPattern Recognition10.1016/j.patcog.2024.111000158:COnline publication date: 1-Feb-2025
https://dl.acm.org/doi/10.1016/j.patcog.2024.111000
Fan WPang KLu PTian C(2024)Making It Tractable to Detect and Correct Errors in GraphsACM Transactions on Database Systems10.1145/370231549:4(1-75)Online publication date: 2-Nov-2024
https://dl.acm.org/doi/10.1145/3702315
Wang JCui ZWang BPan SGao JYin BGao WChua TNgo CKa-Wei Lee RKumar RLauw H(2024)IME: Integrating Multi-curvature Shared and Specific Embedding for Temporal Knowledge Graph CompletionProceedings of the ACM Web Conference 202410.1145/3589334.3645361(1954-1962)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645361
Zhang YKong XShen ZLi JYi QShen GDong B(2024)A survey on temporal knowledge graph embeddingKnowledge-Based Systems10.1016/j.knosys.2024.112454304:COnline publication date: 25-Nov-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.112454
Huang HXie LLiu MLin JShen H(2024)An embedding model for temporal knowledge graphs with long and irregular intervalsKnowledge-Based Systems10.1016/j.knosys.2024.111893296:COnline publication date: 19-Jul-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111893
Yue LRen YZeng YZhang JZeng KWan JZhou M(2024)Complex expressional characterizations learning based on block decomposition for temporal knowledge graph completionKnowledge-Based Systems10.1016/j.knosys.2024.111591290:COnline publication date: 22-Apr-2024
https://dl.acm.org/doi/10.1016/j.knosys.2024.111591
Xu XJia WYan LLu XWang CMa Z(2024)Spatiotemporal knowledge graph completion via diachronic and transregional word embeddingInformation Sciences: an International Journal10.1016/j.ins.2024.120477667:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120477
Li YZhang XZhang BHuang FChen XLu MMa S(2024)SANeInformation Sciences: an International Journal10.1016/j.ins.2024.120430667:COnline publication date: 1-May-2024
https://dl.acm.org/doi/10.1016/j.ins.2024.120430
Wang HYang JYang LGao YDing JZhou XLiu H(2024)MvTuckERInformation Fusion10.1016/j.inffus.2024.102249106:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.inffus.2024.102249
Xie RRuan KHuang BYu WXiao JHuang J(2024)TSA-Net: a temporal knowledge graph completion method with temporal-structural adaptationApplied Intelligence10.1007/s10489-024-05734-154:21(10320-10332)Online publication date: 1-Nov-2024
https://dl.acm.org/doi/10.1007/s10489-024-05734-1
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents