Article

Non-negative multiple matrix factorization

Authors:

Katsuhiko Ishiguro,

Akisato Kimura,

Hiroshi SawadaAuthors Info & Claims

IJCAI '13: Proceedings of the Twenty-Third international joint conference on Artificial Intelligence

Pages 1713 - 1720

Published: 03 August 2013 Publication History

Abstract

Non-negative Matrix Factorization (NMF) is a traditional unsupervised machine learning technique for decomposing a matrix into a set of bases and coefficients under the non-negative constraint. NMF with sparse constraints is also known for extracting reasonable components from noisy data. However, NMF tends to give undesired results in the case of highly sparse data, because the information included in the data is insufficient to decompose. Our key idea is that we can ease this problem if complementary data are available that we could integrate into the estimation of the bases and coefficients. In this paper, we propose a novel matrix factorization method called Non-negative Multiple Matrix Factorization (NMMF), which utilizes complementary data as auxiliary matrices that share the row or column indices of the target matrix. The data sparseness is improved by decomposing the target and auxiliary matrices simultaneously, since auxiliary matrices provide information about the bases and coefficients. We formulate NMMF as a generalization of NMF, and then present a parameter estimation procedure derived from the multiplicative update rule. We examined NMMF in both synthetic and real data experiments. The effect of the auxiliary matrices appeared in the improved NMMF performance. We also confirmed that the bases that NMMF obtained from the real data were intuitive and reasonable thanks to the non-negative constraint.

References

[1]

S.A. Abdallah and M.D. Plumbley. Polyphonic music transcription by non-negative sparse coding of power spectra. In Proc. ISMIR, 2004.

[2]

M. Aharon, M. Elad, and A.M. Bruckstein. On the uniqueness of overcomplete dictionaries, and a practical way to retrieve them. Linear algebra and its applications, 416(1):48-67, 2006.

[3]

D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022, 2003.

[4]

D. Cai, X. He, and J. Han. Graph regularized non-negative matrix factorization for data representation. IEEE Trans., 33:1548-1560, 2011.

[5]

B. Cao, D. Shen, J.T. Sun, X. Wang, Q. Yang, and Z. Chen. Detect and track latent factors with online nonnegative matrix factorization. In Proc. IJCAI, 2007.

[6]

A.T. Cemgil. Bayesian inference for nonnegative matrix factorisation models. Computational Intelligence and Neuroscience, 2009.

[7]

A. Cichocki, R. Zdunek, and S. Amari. Hierarchical ALS algorithms for nonnegative matrix and 3D tensor factorization. In Proc. ICA, 2007.

[8]

A. Cichocki, A. H. Phan R. Zdunek, and S. Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-way Data Analysis, John Wiley. Wiley, 2009.

[9]

O. Dikmen and C. Févotte. Maximum marginal likelihood estimation for nonnegative dictionary learning in the Gamma-Poisson model. IEEE Trans., 2012.

[10]

C. Ding, T. Li, and W. Peng. NMF and PLSI: Equivalence and a hybrid algorithm. In Proc. SIGIR, 2006.

[11]

K. Duh, T. Hirao, A. Kimura, K. Ishiguro, T. Iwata, and C. A. Yeung. Creating stories: Social curation of Twitter messages. In Proc. ICWSM, 2012.

[12]

M. D. Hoffman, D. M. Blei, and P. R. Cook. Bayesian nonparametric matrix factorization for recorded music. In Proc. ICML, 2011.

[13]

T. Hofmann. Probabilistic latent semantic indexing. In Proc. SIGIR, 1999.

[14]

P.O. Hoyer. Non-negative matrix factorization with sparseness constraints. Journal of Machine Learning Research, 5:1457-1469, 2004.

[15]

K. Ishiguro, A. Kimura, and K. Takeuchi. Towards automatic image understanding and mining via social curation. In Proc. ICDM, 2012.

[16]

Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. IEEE, 2009.

[17]

D. D. Lee and H. S. Seung. Learning the parts of objects by non-negative matrix factorization. Nature, 401:788-791, October 1999.

[18]

D. D. Lee, Daniel D., and S. H. Sebastian. Algorithms for non-negative matrix factorization. In Proc. NIPS, 2000.

[19]

J. Lin, R. Snow, and W. Morgan. Smoothing techniques for adaptive online language models: topic tracking in tweet streams. In Proc. SIGKDD, 2011.

[20]

C. Liu, H. Yang, J. Fan, L. He, and Y. Wang. Distributed nonnegative matrix factorization for web-scale dyadic data analysis on MapReduce. In Proc. WWW, 2010.

[21]

H. Liu, Z. Yang, Z. Wu, and X. Li. A-optimal non-negative projection for image representation. In Proc. CVPR, 2012.

[22]

H. Ma, H. Yang, M. R. Lyu, and I. King. Sorec: Social recommendation using probabilistic matrix factorization. In Proc. CIKM, 2008.

[23]

Q. Mei, D. Cai, D. Zhang, and C. Zhai. Topic modeling with network regularization. In Proc. WWW, 2008.

[24]

M. Nakano, J. L. Roux, H. Kameoka, T. Nakamura, N. Ono, and S. Sagayama. Bayesian nonparametric spectrogram modeling based on infinite factorial infinite hidden Markov model. In Proc. WASPAA, 2011.

[25]

J. Noel, S. Sanner, K. Tran, P. Christen, L. Xie, E. V. Bonilla, E. Abbasnejad, and N. D. Penna. New objective functions for social collaborative filtering. In Proc. WWW, 2012.

[26]

S. Purushotham, Y. Liu, and C. C. J. Kuo. Collaborative topic regression with social matrix factorization for recommendation systems. In Proc. ICML, 2012.

[27]

R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In Proc. NIPS, 2008.

[28]

M. N. Schmidt, O. Winther, and L. K. Hansen. Bayesian non-negative matrix factorization. In Proc. ICASSP, 2009.

[29]

P. Smaragdis and J.C. Brown. Non-negative matrix factorization for polyphonic music transcription. In Proc. WASPA, 2003.

[30]

M. Steyvers, P. Smyth, and T. Griffiths. Probabilistic author-topic models for information discovery. In Proc. SIGKDD, 2004.

[31]

C. Wang and D. M. Blei. Collaborative topic modeling for recommending scientific articles. In Proc. SIGKDD, 2011.

[32]

F. Wang, C. Tan, A.C. König, and P. Li. Efficient document clustering via online nonnegative matrix factorizations. In Proc. SIAM, 2011.

[33]

W. Xu, X. Liu, and Y. Gong. Document clustering based on non-negative matrix factorization. In Proc. SIGIR, 2003.

Cited By

He ZLiu JLiu CWang YYin AHuang Y(2019)Dropout non-negative matrix factorizationKnowledge and Information Systems10.1007/s10115-018-1259-x60:2(781-806)Online publication date: 2-Aug-2019
https://dl.acm.org/doi/10.1007/s10115-018-1259-x
Suyama TKishino YShirai YMizutani SSawada H(2018)Interpolation of Missing Data in Sensor Networks Using Nonnegative Matrix FactorizationProceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers10.1145/3267305.3267585(263-266)Online publication date: 8-Oct-2018
https://dl.acm.org/doi/10.1145/3267305.3267585
Nugroho RZhao WYang JParis CNepal S(2017)Using time-sensitive interactions to improve topic derivation in twitterWorld Wide Web10.1007/s11280-016-0417-x20:1(61-87)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1007/s11280-016-0417-x
Show More Cited By

Index Terms

Non-negative multiple matrix factorization
1. Computing methodologies
  1. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

Non-negative Matrix Factorization for Binary Data
IC3K 2015: Proceedings of the International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management

We propose the Logistic Non-negative Matrix Factorization for decomposition of binary data. Binary data

are frequently generated in e.g. text analysis, sensory data, market basket data etc. A common method for

analysing non-negative data is the Non-...
New SVD based initialization strategy for non-negative matrix factorization

We give a new method to determine the rank of the factorization for NMF algorithms.We propose a novel method SVD-NMF to enhance initialization for NMF.The compute process is cheap.Numerical results show that SVD-NMF convergent faster than NNDSVD and ...
Deep alternating non-negative matrix factorisation
Abstract
Non-negative matrix factorisation (NMF) is a promising data-mining technique for non-negative data. NMF achieves feature extraction by factorising the original data matrix into a basis matrix and coding matrix both with non-negative entries. ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

IJCAI '13: Proceedings of the Twenty-Third international joint conference on Artificial Intelligence

August 2013

3266 pages

ISBN:9781577356332

Editor:
Francesca Rossi
University of Padova

Sponsors

The International Joint Conferences on Artificial Intelligence, Inc. (IJCAI)

Publisher

AAAI Press

Publication History

Published: 03 August 2013

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
38
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

He ZLiu JLiu CWang YYin AHuang Y(2019)Dropout non-negative matrix factorizationKnowledge and Information Systems10.1007/s10115-018-1259-x60:2(781-806)Online publication date: 2-Aug-2019
https://dl.acm.org/doi/10.1007/s10115-018-1259-x
Suyama TKishino YShirai YMizutani SSawada H(2018)Interpolation of Missing Data in Sensor Networks Using Nonnegative Matrix FactorizationProceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers10.1145/3267305.3267585(263-266)Online publication date: 8-Oct-2018
https://dl.acm.org/doi/10.1145/3267305.3267585
Nugroho RZhao WYang JParis CNepal S(2017)Using time-sensitive interactions to improve topic derivation in twitterWorld Wide Web10.1007/s11280-016-0417-x20:1(61-87)Online publication date: 1-Jan-2017
https://dl.acm.org/doi/10.1007/s11280-016-0417-x
Zhang GHe MWu HCai GGe JAnjum AZhao X(2016)Non-negative multiple matrix factorization with social similarity for recommender systemsProceedings of the 3rd IEEE/ACM International Conference on Big Data Computing, Applications and Technologies10.1145/3006299.3006323(280-286)Online publication date: 6-Dec-2016
https://dl.acm.org/doi/10.1145/3006299.3006323
Kohjima MMatsubayashi TSawada HBailey JMoffat AAggarwal Cde Rijke MKumar RMurdock VSellis TYu J(2015)Probabilistic Non-negative Inconsistent-resolution Matrices FactorizationProceedings of the 24th ACM International on Conference on Information and Knowledge Management10.1145/2806416.2806636(1855-1858)Online publication date: 17-Oct-2015
https://dl.acm.org/doi/10.1145/2806416.2806636

View Options

View options

Media

Figures

Other

Tables

View Table of Contents