research-article

Structure Learning of Bayesian Network with Latent Variables by Weight-Induced Refinement

Authors:

Weiyi LiuAuthors Info & Claims

Web-KR '14: Proceedings of the 5th International Workshop on Web-scale Knowledge Representation Retrieval & Reasoning

Pages 37 - 44

https://doi.org/10.1145/2663792.2663798

Published: 03 November 2014 Publication History

Abstract

Bayesian network (BN) with latent variables (LVs) provides a concise and straightforward framework for representing and inferring uncertain knowledge with unobservable variables or with regard to missing data. To learn the BN with LVs consistently with the realistic situations, we propose the information theory based concept of existence weight and incorporate it into the clique-based learning method. In line with the challenges when learning BN with LVs, we focus on determining the number of LVs, and determining the relationships between LVs and the observed variables. First, we define the existence weight and propose the algorithms for finding the ε-cliques from the BN without LVs learned from data. Then, we introduce the LV to each ε-clique and adjust the BN structure with LVs. Further, we adjust the value of parameter ε to determine the number of LVs. Experimental results show the feasibility of our method.

References

[1]

Anandkumar, A., Hsu, D., Javanmard, A., and Kakade, S. M. 2013. Learning Linear Bayesian Networks with Latent Variables. In Proc. ICML (Atlanta, GA, USA, June 16 - 21, 2013). ICML '13. JMLR.org, 28, 249--257.

[2]

Bernardo, J. M., Bayarri, M. J., Berger, J. O., Dawid, A. P., Heckerman, D., Smith, A. F. M., and West, M. 2003. The variational Bayesian EM algorithm for incomplete data: with application to scoring graphical model structures. Bayesian statistics, 2003, 7: 453--464.

[3]

Binder, J., Koller, D., Russell, S., and Kanazawa, K. 1997. Adaptive probabilistic networks with hidden variables. Machine Learning, Springer, 1997, 29 (2--3): 213--244. DOI= http://dx.doi.org/10.1023/A:1007421730016.

Digital Library

[4]

Bomze, I. M., Budinich, M., Pardalos, P. M., and Pelillo, M. 1999. The maximum clique problem. Handbook of combinatorial optimization. Springer, 1999, 1--74.

[5]

Chades, I., Carwardine, J., Martin, T. G., Nicol, S., Sabbadin, R., and Buffet, O. 2012. MOMDPs: A Solution for Modelling Adaptive Management Problems. In Proc. AAAI (Toronto, Ontario, Canada, July 22 - 26, 2012). AAAI '12. AAAI Press, 267--273.

[6]

Chen, W. Z., Ji, Z. L., Shen, S., and Yang, Q. 2011. A Whole Page Click Model to Better Interpret Search Engine Click Data. In Proc. AAAI (San Francisco, California, USA, August 07 - 11, 2011). AAAI '11. AAAI Press, 11, 1140--1145.

[7]

Cheng, J., Bell, D., and Liu, W. R. 1997. Learning Bayesian networks from data: An efficient approach based on information theory. In Proc. ACM (Las Vegas, Nevada, November 10 - 14, 1997). ACM, New York, NY, 325--331. DOI= http://doi.acm.org/10.1145/266714.266920.

Digital Library

[8]

Choi, M. J., Tan, V. Y. F., Anandkumar, A., and Willsky, A. S. 2011. Learning Latent Tree Graphical Models. Journal of Machine Learning Research, 2011, 12: 1729--1770.

Digital Library

[9]

Dempster, A. P., Laird, N. M., and Rubin, D. B. 1997. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society. Series B (Methodological), 1977, 1--38.

[10]

Eaton,D. and Murphy, K. 2007. BDAGL: Bayesian DAG learning. http://www.cs.ubc.ca/~murphyk/Software/BDAGL/, 2007.

[11]

Elidan, G., Lotner, N., Friedman, N., and Koller, D. 2000. Discovering hidden variables: A structure-based approach. In Proc. NIPS (Denver, CO, USA, 2000). NIPS '00. MIT Press, 479--485.

[12]

Friedman, N. 1998. The Bayesian structural EM algorithm. In Proc. UAI (Wisconsin, USA, July 24 - 26, 1998). UAI '98. Morgan Kaufmann, 129--138.

Digital Library

[13]

Heckerman, D. 2008. A Tutorial on Learning with Bayesian Networks. Innovations in Bayesian Networks. Springer, 2008, pp. 33--82.

[14]

Heckerman, D., Geiger, D., and Chickering, D. M. 1995. Learning Bayesian networks: The combination of knowledge and statistical data. Machine learning, Springer 1995, 20: 197--243. DOI= http://dx.doi.org/10.1007/BF00994016.

Digital Library

[15]

Ju, H. L., Jin, G. Z., and Namkamori, Y. 2010. Methods for products recommendation using emotional information. In Proc. SMC (Istanbul, Turkey, October 10 - 13, 2010). SMC '10. IEEE, 849--856. DOI= http://dx.doi.org/10.1109/ICSMC. 2010.5641865.

[16]

Karp, R. M. 1972. Reducibility among combinatorial problems. Springer, 1972, 85--103.

[17]

Lanterman, A. D. 2001. Schwarz, Wallace, Rissanen: Intertwining themes in theories of model selection. International Statistical Review, 2001, 69 (2): 185--212.

[18]

Leray, P. and Francois, O. 2004. BNT structure learning package: documentation and experiments. Technical report, Laboratoire PSI, Université et INSA de Rouen, 2004.

[19]

Lu, Z. W. and Peng, Y. X. 2011. Latent Semantic Learning by Efficient Sparse Coding with Hypergraph Regularization. In Proc. AAAI (San Francisco, California, USA, August 07 - 11, 2011). AAAI '11. AAAI Press, 411--416.

[20]

Martin, J. and VanLehn, K. 1995. Discrete factor analysis: Learning hidden variables in Bayesian networks. Technical report, Department of Computer Science, University of Pittsburgh, 1995.

[21]

Mohan, A., Chen, Z., and Weinberger, K. 2011. Web Search Ranking with Initialized Gradient Boosted Regression Trees. Journal of Machine Learning Research, 2011, 14: 77--89.

[22]

Paokanta, P. and Harnpornchai, N. 2012. Risk analysis of Thalassemia using knowledge representation model: Diagnostic Bayesian Networks. In Proc. BHI (Hong Kong, China, January 05 - 07, 2012). BHI '12. IEEE, 155--158. DOI= http://dx.doi.org/10.1109/BHI.2012.6211532.

[23]

Silva, R., Scheine, R., Glymour, C., and Spirtes, P. 2006. Learning the structure of linear latent variable models. Journal of Machine Learning Research, 2006, 7: 191--246.

Digital Library

[24]

Sun, F. T., Griss, M., Mengshoel, O. J., and Yeh, Y. T. 2013. Latent Topic Analysis for Predicting Group Purchasing Behavior on the Social Web. In Proc. UAI 2013 Workshop on Models for Spatial, Temporal and Network Data (Bellvue, Washington, USA, 2013). UAI Application Workshops '13. CEUR-WS.org, 1024, 67--76.

[25]

Wang, Y., Zhang, N. L., and Chen, T. 2008. Latent tree models and approximate inference in Bayesian networks. Journal of Artificial Intelligence Research, 2008, 32: 879--900.

Digital Library

[26]

Xu, Z. X., Zhang, N. L., Wang, Y. Q., Liu, G. P., Xu, J., Liu, T. F., and Liu, A. H. 2013. Statistical Validation of Traditional Chinese Medicine Syndrome Postulates in the Context of Patients with Cardiovascular Disease. The Journal of Alternative and Complementary Medicine, 2013, 18: 1--6.

[27]

Zhang, N. L. and Guo, H. P. 2006. Introduction to Bayesian Networks. Science Press, Beijing, 2006.

Cited By

Liao XLiu X(2024)Hidden Variable Discovery Based on Regression and EntropyMathematics10.3390/math1209137512:9(1375)Online publication date: 30-Apr-2024
https://doi.org/10.3390/math12091375
Wu XYue KDuan LFu X(2024)Learning a Bayesian network with multiple latent variables for implicit relation representationData Mining and Knowledge Discovery10.1007/s10618-024-01012-338:4(1634-1669)Online publication date: 22-Feb-2024
https://doi.org/10.1007/s10618-024-01012-3
Njah HJamoussi SMahdi W(2021)Breaking the curse of dimensionality: hierarchical Bayesian network model for multi-view clusteringAnnals of Mathematics and Artificial Intelligence10.1007/s10472-021-09749-zOnline publication date: 6-May-2021
https://doi.org/10.1007/s10472-021-09749-z
Show More Cited By

Index Terms

Structure Learning of Bayesian Network with Latent Variables by Weight-Induced Refinement
1. Mathematics of computing
  1. Probability and statistics
    1. Statistical paradigms
      1. Statistical graphics

Recommendations

Discrete Bayesian Network Classifiers: A Survey

We have had to wait over 30 years since the naive Bayes model was first introduced in 1960 for the so-called Bayesian network classifiers to resurge. Based on Bayesian networks, these classifiers have many strengths, like model interpretability, ...
Dynamic embeddings for efficient parameter learning of Bayesian network with multiple latent variables
Highlights
- Give a parameter learning method for BN with latent variables on dynamic embeddings.
Abstract
Latent variables (LVs), representing the unobservable abstract concepts, such as patient disease and customer credit, play an important role in the simplification of network structure and improving the interpretability of Bayesian ...
Why BDeu? Regular Bayesian network structure learning with discrete and continuous variables
Abstract
We consider the problem of Bayesian network structure learning (BNSL) from data. In particular, we focus on the score‐based approach rather than the constraint‐based approach and address what score we should use for the purpose. The Bayesian ...
We consider the problem of Bayesian network structure learning (BNSL) from data. In particular, we focus on the score‐based approach rather than the constraint‐based one and address what score we should use for the purpose. The Bayesian Dirichlet ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Web-KR '14: Proceedings of the 5th International Workshop on Web-scale Knowledge Representation Retrieval & Reasoning

November 2014

72 pages

ISBN:9781450316064

DOI:10.1145/2663792

Program Chairs:
Yi Zeng
Chinese Academy of Sciences, China
,
Spyros Kotoulas
IBM Research, Ireland
,
Zhisheng Huang
VU University Amsterdam, The Netherlands

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CIKM '14

Sponsor:

CIKM '14: 2014 ACM Conference on Information and Knowledge Management

November 3, 2014

Shanghai, China

Acceptance Rates

Overall Acceptance Rate 4 of 4 submissions, 100%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
151
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)1

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Liao XLiu X(2024)Hidden Variable Discovery Based on Regression and EntropyMathematics10.3390/math1209137512:9(1375)Online publication date: 30-Apr-2024
https://doi.org/10.3390/math12091375
Wu XYue KDuan LFu X(2024)Learning a Bayesian network with multiple latent variables for implicit relation representationData Mining and Knowledge Discovery10.1007/s10618-024-01012-338:4(1634-1669)Online publication date: 22-Feb-2024
https://doi.org/10.1007/s10618-024-01012-3
Njah HJamoussi SMahdi W(2021)Breaking the curse of dimensionality: hierarchical Bayesian network model for multi-view clusteringAnnals of Mathematics and Artificial Intelligence10.1007/s10472-021-09749-zOnline publication date: 6-May-2021
https://doi.org/10.1007/s10472-021-09749-z
Yu KLiu LLi JChen H(2018)Mining Markov Blankets Without Causal SufficiencyIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2018.282898229:12(6333-6347)Online publication date: Dec-2018
https://doi.org/10.1109/TNNLS.2018.2828982
Gao TJi Q(2016)Constrained local latent variable discoveryProceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence10.5555/3060621.3060828(1490-1496)Online publication date: 9-Jul-2016
https://dl.acm.org/doi/10.5555/3060621.3060828
Njah HJamoussi SMahdi WMasmoudi A(2015)A New Equilibrium Criterion for Learning the Cardinality of Latent VariablesProceedings of the 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI)10.1109/ICTAI.2015.138(958-965)Online publication date: 9-Nov-2015
https://dl.acm.org/doi/10.1109/ICTAI.2015.138
Zeng YKotoulas SHuang ZLi JWang XGarofalakis MSoboroff ISuel TWang M(2014)Web-KR 2014Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge Management10.1145/2661829.2663545(2102-2103)Online publication date: 3-Nov-2014
https://dl.acm.org/doi/10.1145/2661829.2663545

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents