research-article

Knowledge graph fact prediction via knowledge-enriched tensor factorization

Authors:

Konstantinos Kalpakis,

Francis Ferraro,

Tim FininAuthors Info & Claims

Volume 59, Issue C

https://doi.org/10.1016/j.websem.2019.01.004

Published: 01 December 2019 Publication History

Abstract

We present a family of novel methods for embedding knowledge graphs into real-valued tensors. These tensor-based embeddings capture the ordered relations that are typical in the knowledge graphs represented by semantic web languages like RDF. Unlike many previous models, our methods can easily use prior background knowledge provided by users or extracted automatically from existing knowledge graphs. In addition to providing more robust methods for knowledge graph embedding, we provide a provably-convergent, linear tensor factorization algorithm. We demonstrate the efficacy of our models for the task of predicting new facts across eight different knowledge graphs, achieving between 5% and 50% relative improvement over existing state-of-the-art knowledge graph embedding techniques. Our empirical evaluation shows that all of the tensor decomposition models perform well when the average degree of an entity in a graph is high, with constraint-based models doing better on graphs with a small number of highly similar relations and regularization-based models dominating for graphs with relations of varying degrees of similarity.

References

[1]

Ernst P., Siu A., Weikum G., Knowlife: a versatile approach for constructing a large knowledge graph for biomedical sciences, BMC Bioinform. (2015).

[2]

Mintz M., Bills S., Snow R., Jurafsky D., Distant supervision for relation extraction without labeled data, in: Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2-Volume 2, Association for Computational Linguistics, 2009, pp. 1003–1011.

[3]

Auer S., Bizer C., Kobilarov G., Lehmann J., Cyganiak R., Ives Z., Dbpedia: A nucleus for a web of open data, in: International Semantic Web Conference, Springer, 2007, pp. 722–735.

[4]

Bollacker K., Evans C., Paritosh P., Sturge T., Taylor J., Freebase: a collaboratively created graph database for structuring human knowledge, in: Proceedings of the 2008 ACM SIGMOD international conference on Management of data, ACM, 2008, pp. 1247–1250.

[5]

Miller G.A., Wordnet: a lexical database for English, Commun. ACM 38 (11) (1995).

[6]

Ringler D., Paulheim H., One knowledge graph to rule them all? Analyzing the differences between DBpedia, YAGO, Wikidata & co., in: Joint German/Austrian Conference on Artificial Intelligence, Springer, 2017, pp. 366–372.

[7]

DBpedia, Dbpedia sparql endpoint, 2017.

[8]

Bengio Y., Courville A., Vincent P., Representation learning: a review and new perspectives, IEEE Trans. Pattern Anal. Mach. Intell. 35 (8) (2013) 1798–1828.

Digital Library

[9]

M. Nickel, V. Tresp, H.-P. Kriegel, A three-way model for collective learning on multi-relational data, in: Proceedings of the 28th International Conference on Machine Learning, ICML-11, 2011, pp. 809–816.

[10]

Kolda T.G., Bader B.W., Tensor decompositions and applications, SIAM Rev. (2009).

Digital Library

[11]

Franz T., Schultz A., Sizov S., Staab S., Triplerank: Ranking semantic web data by tensor decomposition, in: International Semantic Web Conference, Springer, 2009, pp. 213–228.

[12]

Krompass D., Baier S., Tresp V., Type-constrained representation learning in knowledge graphs, in: International Semantic Web Conference, Springer, 2015.

[13]

D. Krompass, M. Nickel, X. Jiang, V. Tresp, Non-negative tensor factorization with RESCAL, in: Tensor Methods for Machine Learning, ECML Workshop, 2013.

[14]

A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O. Yakhnenko, Translating embeddings for modeling multi-relational data, in: NIPS, 2013, pp. 2787–2795.

[15]

Yang B., Yih W.-T., He X., Gao J., Deng L., Embedding entities and relations for learning and inference in knowledge bases, 2014, arXiv preprint arXiv:1412.6575.

[16]

T. Trouillon, J. Welbl, S. Riedel, É. Gaussier, G. Bouchard, Complex embeddings for simple link prediction, in: International Conference on Machine Learning, 2016, pp. 2071–2080.

[17]

T. Demeester, T. Rocktäschel, S. Riedel, Lifted Rule Injection for Relation Embeddings, in: Conference on Empirical Methods in Natural Language Processing, 2016, pp. 1389–1399.

[18]

P. Minervini, T. Demeester, T. Rocktäschel, S. Riedel, Adversarial sets for regularising neural link predictors, in: 33rd Conference on Uncertainty in Artificial Intelligence, 2017, pp. 1–10.

[19]

Lengerich B.J., Maas A.L., Potts C., Retrofitting distributional embeddings to knowledge graphs with functional relations, 2017, arXiv preprint arXiv:1708.00112.

[20]

Minervini P., Costabello L., Muñoz E., Nováček V., Vandenbussche P.-Y., Regularizing knowledge graph embeddings via equivalence and inversion axioms, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2017, pp. 668–683.

[21]

Padia A., Kalpakis K., Finin T., Inferring relations in knowledge graphs with tensor decompositions, in: International Conference on Big Data, IEEE, 2016, pp. 4020–4022.

[22]

Paulheim H., Knowledge graph refinement: A survey of approaches and evaluation methods, Semantic Web 8 (3) (2017) 489–508.

Digital Library

[23]

Nickel M., Murphy K., Tresp V., Gabrilovich E., A review of relational machine learning for knowledge graphs, Proc. IEEE 104 (1) (2016) 11–33.

[24]

Bordes A., Glorot X., Weston J., Bengio Y., A semantic matching energy function for learning with multi-relational data, Mach. Learn. (2014) 233–259.

[25]

Socher R., Chen D., Manning C.D., Ng A., Reasoning with neural tensor networks for knowledge base completion, in: Advances in Neural Information Processing Systems, 2013, pp. 926–934.

[26]

Z. Wang, J. Zhang, J. Feng, Z. Chen, Knowledge graph embedding by translating on hyperplanes, in: AAAI, vol. 14, 2014, pp. 1112–1119.

[27]

M. Nickel, L. Rosasco, T.A. Poggio, et al. Holographic embeddings of knowledge graphs, in: AAAI, vol. 2, 2016, pp. 3–2.

[28]

Hayashi K., Shimbo M., On the equivalence of holographic and complex embeddings for link prediction, 2017, arXiv preprint arXiv:1702.05563.

[29]

S. Guo, Q. Wang, L. Wang, B. Wang, L. Guo, Knowledge graph embedding with iterative guidance from soft rules, AAAI, 2018.

[30]

Galárraga L.A., Teflioudi C., Hose K., Suchanek F., AMIE: association rule mining under incomplete evidence in ontological knowledge bases, in: Proceedings of the 22nd International Conference on World Wide Web, ACM, 2013, pp. 413–422.

[31]

Miettinen P., Boolean tensor factorizations, in: 11th International Conference on Data Mining, ICDM, IEEE, 2011, pp. 447–456.

[32]

Erdos D., Miettinen P., Discovering facts with boolean tensor tucker decomposition, in: Proceedings of the 22nd International Conference on Conference on Information & Knowledge Management, ACM, 2013, pp. 1569–1572.

[33]

Nickel M., Tresp V., Kriegel H.-P., Factorizing YAGO: scalable machine learning for linked data, in: Proceedings of the 21st International Conference on World Wide Web, ACM, 2012, pp. 271–280.

[34]

Minervini P., d’Amato C., Fanizzi N., Esposito F., Leveraging the schema in latent factor models for knowledge graph completion, in: Proceedings of the 31st Annual ACM Symposium on Applied Computing, SAC ’16, ACM, New York, NY, USA, 2016, pp. 327–332,. URL http://doi.acm.org/10.1145/2851613.2851841.

Digital Library

[35]

L. Han, T. Finin, A. Joshi, D. Cheng, Querying RDF data with text annotated graphs, in: 27th International Conference on Scientific and Statistical Database Management, 2015.

[36]

J.M. Gawron, Improving sparse word similarity models with asymmetric measures, in: ACL (2), 2014, pp. 296–301.

[37]

Parikh N., Boyd S., Proximal algorithms, Found. Trends Optim. 1 (3) (2014) 123–231.

[38]

Bader B.W., Harshman R.A., Kolda T.G., Temporal analysis of semantic graphs using ASALSAN, in: Proceedings of the 7th International Conference on Data Mining, IEEE, 2007, pp. 33–42.

[39]

D. Kingma, J. Ba, Adam: A method for stochastic optimization, in: Proceedings of the Third International Conference on Learning Representations, 2014.

[40]

Baker C.F., Fillmore C.J., Lowe J.B., The Berkeley framenet project, in: 36th Annual Meeting of the ACL and 17th Int. Conf. on Computational Linguistics, ACL, 1998, pp. 86–90.

[41]

Nickel M., Tensor factorization library, 2013, (Online; Accessed 10 August 2017).

[42]

Harshman R.A., Lundy M.E., PARAFAC: Parallel Factor analysis, Comput. Statist. Data Anal. (1994).

[43]

Bro R., PARAFAC. Tutorial And applications, Chemometr. Intell. Lab. Syst. 38 (1997) 149–171.

[44]

Kemp C., Tenenbaum J.B., Griffiths T.L., Yamada T., Ueda N., Learning systems of concepts with an infinite relational model, in: Proceedings of the 21st National Conference on Artificial Intelligence, AAAI, 2006, pp. 381–388.

[45]

Dettmers T., Minervini P., Stenetorp P., Riedel S., Convolutional 2D knowledge graph embeddings, 2018, arXiv preprint arXiv:1707.01476, Extended AAAI18 paper.

[46]

K. Toutanova, D. Chen, Observed versus latent features for knowledge base and text inference, in: Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality, 2015, pp. 57–66.

[47]

R. Kadlec, O. Bajgar, J. Kleindienst, Knowledge base completion: Baselines strike back, in: Proceedings of the 2nd Workshop on Representation Learning for NLP, 2017, pp. 69–74.

[48]

Baker C.F., Fillmore C.J., Lowe J.B., The Berkeley framenet project, in: Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics-Volume 1, Association for Computational Linguistics, 1998, pp. 86–90.

[49]

K. Toutanova, D. Chen, P. Pantel, H. Poon, P. Choudhury, M. Gamon, Representing text for joint embedding of text and knowledge bases, in: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, 2015, pp. 1499–1509.

[50]

Finin T., Lawrie D., McNamee P., Mayfield J., Oard D., Peng N., Gao N., Lin Y.-C., MacKin J., Dowd T., HLTCOE Participation in TAC KBP 2015: Cold start and TEDL, in: 8th Text Analysis Conference, NIST, 2015.

[51]

Syed Z., Padia A., Mathews M.L., Finin T., Joshi A., UCO: A Unified cybersecurity ontology, in: Proceedings of the AAAI Workshop on Artificial Intelligence for Cyber Security, AAAI Press, 2016.

[52]

Narayanan S.N., Ganesan A., Joshi K.P., Oates T., Joshi A., Finin T., Early detection of cybersecurity threats using collaborative cognition, in: Proceedings of the 4th IEEE International Conference on Collaboration and Internet Computing (CIC), IEEE, 2018.

[53]

A. Padia, Cleaning noisy knowledge graphs, in: Proceedings of the Doctoral Consortium At the 16th International Semantic Web Conference, vol. 1962, orgname = CEUR Workshop Proceedings, 2017.

[54]

Nickel M., Tensor factorization for relational learning, (Ph.D. thesis) Ludwig-Maximilians-Universität München, 2013.

[55]

Grippof L., Sciandrone M., Globally convergent block-coordinate techniques for unconstrained optimization, Optim. Methods Softw. 10 (4) (1999).

Cited By

Liu CZhang HZhang JZhang ZYuan P(2023)Design of a Learning Path Recommendation System Based on a Knowledge GraphInternational Journal of Information and Communication Technology Education10.4018/IJICTE.31996219:1(1-18)Online publication date: 17-Mar-2023
https://dl.acm.org/doi/10.4018/IJICTE.319962
Zulaika UAlmeida ALópez-de-Ipiña D(2023)Regularized online tensor factorization for sparse knowledge graph embeddingsNeural Computing and Applications10.1007/s00521-022-07796-z35:1(787-797)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s00521-022-07796-z

Index Terms

Knowledge graph fact prediction via knowledge-enriched tensor factorization

Index terms have been assigned to the content through auto-classification.

Recommendations

FedE: Embedding Knowledge Graphs in Federated Setting
IJCKG '21: Proceedings of the 10th International Joint Conference on Knowledge Graphs

Knowledge graphs (KGs) become widespread and many organizations construct as well as maintain their own knowledge graphs. Same as the data isolation which has been a long-standing problem, knowledge graph isolation is common in real knowledge graph ...
CP Tensor Factorization for Knowledge Graph Completion
Knowledge Science, Engineering and Management
Abstract
The problem of incomplete knowledge caused by the lack of relations in large-scale knowledge graphs increases the difficulty of downstream application tasks. Predicting the missing relations between entities according to the existing facts is the ...
Regularized online tensor factorization for sparse knowledge graph embeddings
Abstract
Knowledge Graphs represent real-world facts and are used in several applications; however, they are often incomplete and have many missing facts. Link prediction is the task of completing these missing facts from existing ones. Embedding models ... $_{}$

Comments

Information & Contributors

Information

Published In

cover image Web Semantics: Science, Services and Agents on the World Wide Web

Web Semantics: Science, Services and Agents on the World Wide Web Volume 59, Issue C

Dec 2019

121 pages

ISSN:1570-8268

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 01 December 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 20 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu CZhang HZhang JZhang ZYuan P(2023)Design of a Learning Path Recommendation System Based on a Knowledge GraphInternational Journal of Information and Communication Technology Education10.4018/IJICTE.31996219:1(1-18)Online publication date: 17-Mar-2023
https://dl.acm.org/doi/10.4018/IJICTE.319962
Zulaika UAlmeida ALópez-de-Ipiña D(2023)Regularized online tensor factorization for sparse knowledge graph embeddingsNeural Computing and Applications10.1007/s00521-022-07796-z35:1(787-797)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1007/s00521-022-07796-z

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents