skip to main content
research-article

Multi-view unsupervised feature selection with tensor robust principal component analysis and consensus graph learning

Published: 01 September 2023 Publication History

Highlights

A unified multi-view unsupervised feature selection framework based on robust principal component analysis is proposed.
We adaptively obtain the high-quality consensus local geometric structure and reliable pseudo labels to select discriminative features.
We perform extensive experiments on six multi-view datasets and the comparison results confirm the superiority of our method.

Abstract

Recently, multi-view unsupervised feature selection has attracted much attention due to its efficiency and better interpretability in processing high-dimensional multi-view datasets. Most existing methods rely on the constructed similarity matrices to obtain reliable pseudo labels to guide the feature selection. However, the considerable adverse noise in the raw data inevitably impedes the exploration of true underlying similarity structures. Besides, the inter-view correlations are often ignored during the common representation learning, which limits the effective fusion of the essential information from multiple views. To solve these issues, we design a novel robust multi-view unsupervised feature selection framework. Specifically, our method seeks a set of noise-free view-specific similarity matrices by leveraging tensor robust principal component analysis, where the high-order connections among different views are well exploited through the constructed low-rank tensor. Meanwhile, a high-quality consensus similarity matrix is adaptively learned from the view-specific representations within the same unified framework to capture the shared local structures. To enhance the discriminative ability of the feature selection matrix, we further impose a rank constraint on the consensus similarity matrix to obtain reliable pseudo cluster indicators. We present an efficient optimization algorithm ground on the alternating direction method of multipliers to solve the proposed model. Experimental results on six multi-view datasets confirm the superiority of our method.

References

[1]
W. Chu, H. Xue, C. Yao, D. Cai, Sparse coding guided spatiotemporal feature learning for abnormal event detection in large videos, IEEE Trans. Multimed. 21 (1) (2018) 246–255,.
[2]
D. Hong, L. Gao, N. Yokoya, J. Yao, J. Chanussot, Q. Du, B. Zhang, More diverse means better: multimodal deep learning meets remote-sensing imagery classification, IEEE Trans. Geosci. Remote Sens. 59 (5) (2020) 4340–4354,.
[3]
Y. Guo, Z. Zhang, F. Tang, Feature selection with kernelized multi-class support vector machine, Pattern Recognit. 117 (2021) 107988,.
[4]
D. Hong, L. Gao, J. Yao, B. Zhang, A. Plaza, J. Chanussot, Graph convolutional networks for hyperspectral image classification, IEEE Trans. Geosci. Remote Sens. 59 (7) (2020) 5966–5978,.
[5]
R. Shang, Y. Meng, W. Wang, F. Shang, L. Jiao, Local discriminative based sparse subspace learning for feature selection, Pattern Recognit. 92 (2019) 219–230,.
[6]
Y. Hu, J.-X. Liu, Y.-L. Gao, J. Shang, DSTPCA: double-sparse constrained tensor principal component analysis method for feature selection, IEEE/ACM Trans. Comput. Biol. Bioinf. 18 (4) (2019) 1481–1491,.
[7]
X. Wu, X. Xu, J. Liu, H. Wang, B. Hu, F. Nie, Supervised feature selection with orthogonal regression and feature weighting, IEEE Trans. Neural Netw. Learn. Syst. 32 (5) (2020) 1831–1838,.
[8]
R. Sheikhpour, M.A. Sarram, S. Gharaghani, M.A.Z. Chahooki, A survey on semi-supervised feature selection methods, Pattern Recognit. 64 (2017) 141–158,.
[9]
H. Zhang, D. Wu, F. Nie, R. Wang, X. Li, Multilevel projections with adaptive neighbor graph for unsupervised multi-view feature selection, Inf. Fusion 70 (2021) 129–140,.
[10]
S. Wang, W. Pedrycz, Q. Zhu, W. Zhu, Subspace learning for unsupervised feature selection via matrix factorization, Pattern Recognit. 48 (1) (2015) 10–19,.
[11]
X. He, D. Cai, P. Niyogi, Laplacian score for feature selection, Advances in Neural Information Processing Systems, vol. 18, 2005, pp. 507–514.
[12]
F. Nie, W. Zhu, X. Li, Unsupervised feature selection with structured graph optimization, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30, 2016, pp. 1302–1308,.
[13]
Y. Feng, J. Xiao, Y. Zhuang, X. Liu, Adaptive unsupervised multi-view feature selection for visual concept recognition, Asian Conference on Computer Vision, vol. 7724, Springer, 2012, pp. 343–357,.
[14]
J. Tang, X. Hu, H. Gao, H. Liu, Unsupervised feature selection for multi-view data in social media, Proceedings of the 2013 SIAM International Conference on Data Mining, SIAM, 2013, pp. 270–278,.
[15]
Z. Wang, Y. Feng, T. Qi, X. Yang, J.J. Zhang, Adaptive multi-view feature selection for human motion retrieval, Signal Process. 120 (2016) 691–701,.
[16]
C. Hou, F. Nie, H. Tao, D. Yi, Multi-view unsupervised feature selection with adaptive similarity and view weight, IEEE Trans. Knowl. Data Eng. 29 (9) (2017) 1998–2011,.
[17]
X. Bai, L. Zhu, C. Liang, J. Li, X. Nie, X. Chang, Multi-view feature selection via nonnegative structured graph learning, Neurocomputing 387 (2020) 110–122,.
[18]
C. Tang, X. Zheng, X. Liu, W. Zhang, J. Zhang, J. Xiong, L. Wang, Cross-view locality preserved diversity and consensus learning for multi-view unsupervised feature selection, IEEE Trans. Knowl. Data Eng. 34 (10) (2021) 4705–4716,.
[19]
M. Wang, Q. Wang, D. Hong, S.K. Roy, J. Chanussot, Learning tensor low-rank representation for hyperspectral anomaly detection, IEEE Trans. Cybern. 53 (1) (2022) 679–691,.
[20]
Y.-Y. Zhao, C.-N. Jiao, M.-L. Wang, J.-X. Liu, J. Wang, C.-H. Zheng, HTRPCA: hypergraph regularized tensor robust principal component analysis for sample clustering in tumor omics data, Interdiscip. Sci. 14 (1) (2022) 22–33,.
[21]
H. Yuan, J. Li, Y. Liang, Y.Y. Tang, Multi-view unsupervised feature selection with tensor low-rank minimization, Neurocomputing 487 (2022) 75–85,.
[22]
L. Yuan, G. Yang, Q. Xu, T. Lu, Discriminative feature selection with directional outliers correcting for data classification, Pattern Recognit. 126 (2022) 108541,.
[23]
J. Miao, T. Yang, L. Sun, X. Fei, L. Niu, Y. Shi, Graph regularized locally linear embedding for unsupervised feature selection, Pattern Recognit. 122 (2022) 108299,.
[24]
R. Huang, Z. Wu, Multi-label feature selection via manifold regularization and dependence maximization, Pattern Recognit. 120 (2021) 108149,.
[25]
Z. Zhao, H. Liu, Spectral feature selection for supervised and unsupervised learning, Proceedings of the 24th International Conference on Machine Learning, 2007, pp. 1151–1157,.
[26]
F. Wang, L. Zhu, J. Li, H. Chen, H. Zhang, Unsupervised soft-label feature selection, Knowledge-Based Syst. 219 (2021) 106847,.
[27]
Z. Li, Y. Yang, J. Liu, X. Zhou, H. Lu, Unsupervised feature selection using nonnegative spectral analysis, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 26, 2012, pp. 1026–1032,.
[28]
X. Dong, L. Zhu, X. Song, J. Li, Z. Cheng, Adaptive collaborative similarity learning for unsupervised multi-view feature selection, Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13–19, 2018, Stockholm, Sweden, 2018, pp. 2064–2070,.
[29]
Z. Zhang, G. Ely, S. Aeron, N. Hao, M. Kilmer, Novel methods for multilinear data completion and de-noising based on tensor-SVD, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 3842–3849,.
[30]
J. Wu, Z. Lin, H. Zha, Essential tensor learning for multi-view spectral clustering, IEEE Trans. Image Process. 28 (12) (2019) 5910–5922,.
[31]
N. Shahid, V. Kalofolias, X. Bresson, M. Bronstein, P. Vandergheynst, Robust principal component analysis on graphs, Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 2812–2820,.
[32]
C. Lu, J. Feng, Y. Chen, W. Liu, Z. Lin, S. Yan, Tensor robust principal component analysis with a new tensor nuclear norm, IEEE Trans. Pattern Anal. Mach. Intell. 42 (4) (2019) 925–938,.
[33]
R. Xia, Y. Pan, L. Du, J. Yin, Robust multi-view spectral clustering via low-rank and sparse decomposition, Proceedings of the AAAI Conference on Artificial Intelligence, vol. 28, 2014, pp. 2149–2155,.
[34]
Y. Xie, D. Tao, W. Zhang, Y. Liu, L. Zhang, Y. Qu, On unifying multi-view self-representations for clustering by tensor multi-rank minimization, Int. J. Comput. Vis. 126 (11) (2018) 1157–1179,.
[35]
X. Guo, Robust subspace segmentation by simultaneously learning data representations and their affinity matrix, Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015, pp. 3547–3553.
[36]
F. Nie, X. Wang, H. Huang, Clustering and projected clustering with adaptive neighbors, Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2014, pp. 977–986,.
[37]
K. Fan, On a theorem of Weyl concerning eigenvalues of linear transformations i, Proc. Natl. Acad. Sci. 35 (11) (1949) 652–655,.
[38]
W. Hu, D. Tao, W. Zhang, Y. Xie, Y. Yang, The twist tensor nuclear norm for video completion, IEEE Trans. Neural Netw. Learn. Syst. 28 (12) (2016) 2961–2973,.
[39]
G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, Y. Ma, Robust recovery of subspace structures by low-rank representation, IEEE Trans. Pattern Anal. Mach. Intell. 35 (1) (2012) 171–184,.
[40]
J. Huang, F. Nie, H. Huang, A new simplex sparse learning model to measure data similarity for clustering, Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015, pp. 3569–3575.
[41]
Y. Liu, R. Jin, L. Yang, Semi-supervised multi-label learning by constrained non-negative matrix factorization, AAAi, vol. 6, 2006, pp. 421–426.
[42]
J. Winn, N. Jojic, Locus: learning object classes with unsupervised segmentation, Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1, vol. 1, IEEE, 2005, pp. 756–763,.
[43]
M. van Breukelen, R.P.W. Duin, D.M.J. Tax, J.E. Den Hartog, Handwritten digit recognition by combined classifiers, Kybernetika 34 (4) (1998) 381–386,.
[44]
E. Douglas-Cowie, R. Cowie, M. Schröder, A new emotion database: considerations, sources and scope, ISCA Tutorial and Research Workshop (ITRW) on Speech and Emotion, 2000, pp. 39–44.
[45]
J. Liu, Y. Yang, M. Shah, Learning semantic visual vocabularies using diffusion distance, 2009 IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2009, pp. 461–468,.
[46]
P.A. Estévez, M. Tesmer, C.A. Perez, J.M. Zurada, Normalized mutual information feature selection, IEEE Trans. Neural Netw. 20 (2) (2009) 189–201,.
[47]
S. Huang, Z. Kang, Z. Xu, Auto-weighted multi-view clustering via deep matrix decomposition, Pattern Recognit. 97 (2020) 107015,.
[48]
Q. Wang, X. Jiang, M. Chen, X. Li, Autoweighted multiview feature selection with graph optimization, IEEE Trans. Cybern. 52 (12) (2021) 12966–12977,.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Pattern Recognition
Pattern Recognition  Volume 141, Issue C
Sep 2023
638 pages

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 September 2023

Author Tags

  1. Multi-view unsupervised feature selection
  2. Low-rank tensor learning
  3. Spectral embedding
  4. Robust sparse regression model

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 25 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

View options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media