Joint Latent Space and Label Inference Estimation with Adaptive Fused Data and Label Graphs
Authors: 
A. Baradaaji, F. DornaikaAuthors Info & Claims
A. Baradaaji, F. DornaikaAuthors Info & ClaimsArticle No.: 62, Pages 1 - 25
Abstract
Recently, structured computing has become an interesting topic in the world of artificial intelligence, especially in the field of machine learning, as most researchers focus on the development of graph-based semi-supervised learning models. In this article, we present a new framework for graph-based semi-supervised learning. We present a powerful method for simultaneous label inference and linear transform estimation. The targeted linear transformation is used to obtain a discriminant subspace. To improve semi-supervised learning, our framework focuses on exploiting the data structure and soft labels of the available unlabeled samples. In the iterative optimization scheme used, the prior estimation of the label increases the supervision information indirectly through an introduced informative matrix called the label graph, thus avoiding the use of hard confidence-based decisions as used in self-supervised methods. In addition, the estimation of labels and projected data is made more robust by using smoothing concepts based on hybrid graphs. For each type of smoothing, the hybrid graph is an adaptive fusion of the two graphs encoding the similarity of the data and the similarity of the labels. The proposed method leads to an improved discriminative linear transformation. Several experimental results on real image datasets confirm the effectiveness of the proposed method. They also show superior performance compared to semi-supervised methods that use integration and label inference simultaneously.
References
[1]
Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. 2006. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research 7 (2006), 2399–2434.
[2]
Deng Cai, Xiaofei He, and Jiawei Han. 2007. Semi-supervised discriminant analysis. In IEEE International Conference on Computer Vision.
[3]
F. Dornaika, A. Baradaaji, and Y. El Traboulsi. 2021. Semi-supervised classification via simultaneous label and discriminant embedding estimation. Information Sciences 546 (2021), 146–165.
[4]
Fadi Dornaika, Abdullah Baradaaji, and Youssof El Traboulsi. 2022. Joint label inference and discriminant embedding. IEEE Transactions on Neural Networks and Learning Systems 33 (2022), 4413–4423.
[5]
Fadi Dornaika and Youssof El Traboulsi. 2016. Learning flexible graph-based semi-supervised embedding. IEEE Transactions on Cybernetics 46, 1 (2016), 206–218.
[6]
F. Dornaika and Y. El Traboulsi. 2017. Matrix exponential based semi-supervised discriminant embedding. Pattern Recognition 61 (2017), 92–103.
[7]
F. Dornaika and Y. El Traboulsi. 2019. Joint sparse graph and flexible embedding for graph-based semi-supervised learning. Neural Networks 114 (2019), 91–95.
[8]
Youssof El Traboulsi, Fadi Dornaika, and Ammar Assoum. 2015. Kernel flexible manifold embedding for pattern classification. Neurocomputing 167 (2015), 517–527.
[9]
Chao Han, Deyun Zhou, Yu Xie, Yu Lei, and Jiao Shi. 2021. Label propagation with multi-stage inference for visual domain adaptation. Knowledge-based Systems 216 (2021), 106809.
[10]
Fang He, Feiping Nie, Rong Wang, Weimin Jia, Fenggan Zhang, and Xuelong Li. 2020. Semisupervised band selection with graph optimization for hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing 59, 12 (2020), 10298–10311.
[11]
Fang He, Feiping Nie, Rong Wang, Xuelong Li, and Weimin Jia. 2019. Fast semisupervised learning with bipartite graph for large-scale data. IEEE transactions on neural networks and learning systems 31, 2 (2019), 626–638.
[12]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’16). 770–778.
[13]
Chenping Hou, Feiping Nie, Xuelong Li, Dongyun Yi, and Yi Wu. 2014. Joint embedding learning and sparse regression: A framework for unsupervised feature selection. IEEE Transactions on Cybernetics 44, 6 (2014), 793–804.
[14]
Hong Huang, Jiamin Liu, and Yinsong Pan. 2012. Semi-supervised marginal Fisher analysis for hyperspectral image classification. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 3 (2012), 377–382.
[15]
Sheng Huang, Ahmed Elgammal, Luwen Huangfu, Dan Yang, and Xiaohong Zhang. 2014. Globality-locality preserving projections for biometric data dimensionality reduction. In IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW’14).
[16]
Shudong Huang, Zhao Kang, Ivor W. Tsang, and Zenglin Xu. 2019. Auto-weighted multi-view clustering via kernelized graph learning. Pattern Recognition 88 (2019), 174–184.
[17]
Yi Huang, Dong Xu, and Feiping Nie. 2012. Semi-supervised dimension reduction using trace ratio criterion. IEEE Transactions on Neural Networks and Learning Systems 23, 3 (2012), 519–526.
[18]
Jieru Jia, Qiuqi Ruan, Yi Jin, Gaoyun An, and Shiming Ge. 2020. View-specific subspace learning and re-ranking for semi-supervised person re-identification. Pattern Recognition 108 (2020), 107568.
[19]
Meng Jian and Cheolkon Jung. 2021. Semi-supervised kernel matrix learning using adaptive constraint-based seed propagation. Pattern Recognition 112 (2021), 107750.
[20]
Zhao Kang, Chong Peng, Qiang Cheng, Xinwang Liu, Xi Peng, Zenglin Xu, and Ling Tian. 2021. Structured graph learning for clustering and semi-supervised classification. Pattern Recognition 110 (2021), 107627.
[21]
M. Karasuyama and H. Mamitsuka. 2013. Multiple graph label propagation by sparse integration. IEEE Transactions on Neural Networks and Learning Systems 24, 12 (2013), 1999–2012.
[22]
Thomas N. Kipf and Max Welling. 2017. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations.
[23]
Wei Liu and Shih-Fu Chang. 2009. Robust multi-class transductive learning with graphs. In IEEE Conference on Computer Vision and Pattern Recognition, 2009 (CVPR’09). IEEE, 381–388.
[24]
Zheng Liu, Shiluo Huang, Wei Jin, and Ying Mu. 2021. Broad learning system for semi-supervised learning. Neurocomputing 444 (2021), 38–47.
[25]
L. V. D. Maaten and G. Hinton. 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 2605 (2008), 2579–2605.
[26]
Feiping Nie, Guohao Cai, Jing Li, and Xuelong Li. 2018. Auto-weighted multi-view learning for image clustering and semi-supervised classification. IEEE Transactions on Image Processing 27, 3 (2018), 1501–1511.
[27]
F. Nie, G. Cai, and X. Li. 2017. Multi-view clustering and semi-supervised classification with adaptive neighbours. In 31st AAAI Conference on Artificial Intelligence.
[28]
Feiping Nie, Xia Dong, and Xuelong Li. 2020. Unsupervised and semisupervised projection with graph optimization. IEEE Transactions on Neural Networks and Learning Systems 32, 4 (2020), 1547–1559.
[29]
F. Nie, J. Li, and X. Li. 2016. Parameter-free auto-weighted multiple graph learning: A framework for multiview clustering and semi-supervised classification. In International Joint Conference on Artificial Intelligence (IJCAI’16).
[30]
F. Nie, Z. Wang, R. Wang, and X. Li. 2020. Submanifold-preserving discriminant analysis with an auto-optimized graph. IEEE Transactions on Cybernetics 50, 8 (2020), 3682–3695.
[31]
Feiping Nie, Zheng Wang, Rong Wang, and Xuelong Li. 2021. Adaptive local embedding learning for semi-supervised dimensionality reduction. IEEE Transactions on Knowledge and Data Engineering 34, 10 (2021), 4609–4621.
[32]
F. Nie, S. Xiang, Y. Jia, C. Zhang, and S. Yan. 2008. Trace ratio criterion for feature selection. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence.
[33]
Feiping Nie, Dong Xu, Ivor Wai-Hung Tsang, and Changshui Zhang. 2010. Flexible manifold embedding: A framework for semi-supervised and unsupervised dimension reduction. IEEE Transactions on Image Processing 19, 7 (2010), 1921–1932.
[34]
Haiqi Pan and Zhao Kang. 2018. Robust graph learning for semi-supervised classification. In International Conference on Intelligent Human-machine Systems and Cybernetics.
[35]
Lishan Qiao, Songcan Chen, and Xiaoyang Tan. 2010. Sparsity preserving discriminant analysis for single training image face recognition. Pattern Recognition Letters 31, 5 (2010), 422–429.
[36]
Shougang Ren, Xingjian Gu, Peisen Yuan, and Huanliang Xu. 2018. An iterative paradigm of joint feature extraction and labeling for semi-supervised discriminant analysis. Neurocomputing 273 (2018), 466–480.
[37]
Shaojun Shi, Feiping Nie, Rong Wang, and Xuelong Li. 2020. Auto-weighted multi-view clustering via spectral embedding. Neurocomputing 399 (2020), 369–379.
[38]
C. Sousa, S. Rezende, and G. Batista. 2013. Influence of graph construction on semi-supervised learning. In European Conference on Machine Learning. 160–175.
[39]
Y. El Traboulsi and F. Dornaika. 2018. Flexible semi-supervised embedding based on adaptive loss regression: Application to image categorization. Information Sciences 444 (2018), 1–9.
[40]
Thijs van der Horst, Maarten Löffler, and Frank Staals. 2022. Chromatic k-nearest neighbor queries. arXiv preprint arXiv:2205.00277 (2022).
[41]
Sheng Wang, Jianfeng Lu, Xingjian Gu, Haishun Du, and Jingyu Yang. 2016. Semi-supervised linear discriminant analysis for dimension reduction and classification. Pattern Recognition 57 (2016), 179–189.
[42]
Jie Wen, Xiaozhao Fang, Yong Xu, Chunwei Tian, and Lunke Fei. 2018. Low-rank representation with adaptive graph regularization. Neural Networks 108 (2018), 83–96.
[43]
Wenzhu Yan, Quansen Sun, Huaijiang Sun, and Yanmeng Li. 2020. Semi-supervised learning framework based on statistical analysis for image set classification. Pattern Recognition 107 (2020), 107500.
[44]
Shuyi Yang, Dino Ienco, Roberto Esposito, and Ruggero G. Pensa. 2021. ESA: A generic framework for semi-supervised inductive learning. Neurocomputing 447 (2021), 102–117.
[45]
Guoxian Yu, Guoji Zhang, Carlotta Domeniconi, Zhiwen Yu, and Jane You. 2012. Semi-supervised classification based on random subspace dimensionality reduction. Pattern Recognition 45, 3 (2012), 1119–1135.
[46]
Mingbo Zhao, Yue Zhang, Zhao Zhang, Jiao Liu, and Weijian Kong. 2019. ALG: Adaptive low-rank graph regularization for scalable semi-supervised and unsupervised learning. Neurocomputing 370 (2019), 16–27.
[47]
Dengyong Zhou, Olivier Bousquet, Thomas Navin Lal, Jason Weston, and Bernhard Schölkopf. 2004. Learning with local and global consistency. Advances in Neural Information Processing Systems 16, 16 (2004), 321–328.
[48]
Ruifeng Zhu, Fadi Dornaika, and Yassine Ruichek. 2019. Joint graph based embedding and feature weighting for image classification. Pattern Recognition 93 (2019), 458–469.
[49]
Ruifeng Zhu, Fadi Dornaika, and Yassine Ruichek. 2021. Inductive semi-supervised learning with Graph Convolution based regression. Neurocomputing 434 (2021), 315–322.
[50]
Xiaojin Zhu, Zoubin Ghahramani, and John Lafferty. 2003. Semi-supervised learning using Gaussian fields and harmonic functions. In International Conference on Machine Learning. 912–919.
[51]
Najmeh Ziraki, Fadi Dornaika, and Alireza Bosaghzadeh. 2022. Multiple-view flexible semi-supervised classification through consistent graph construction and label propagation. Neural Networks 146 (2022), 174–180.
Index Terms
- Joint Latent Space and Label Inference Estimation with Adaptive Fused Data and Label Graphs
Recommendations
Simultaneous label inference and discriminant projection estimation through adaptive self-taught graphs
AbstractProcessing structured data has become an interesting topic in recent years. The development of graph-based semi-supervised learning models has attracted much attention from machine learning researchers. In this paper, we present a novel approach ...
Highlights- A flexible and adaptive graph-based semi-supervised embedding is proposed.
- The soft labels and the linear transformation are simultaneously estimated.
- Smoothness of labels and projections is imposed using two hybrid graphs.
- The ...
A unified semi-supervised model with joint estimation of graph, soft labels and latent subspace
AbstractSince manually labeling images is expensive and labor intensive, in practice we often do not have enough labeled images to train an effective classifier for the new image classification tasks. The graph-based SSL methods have received more ...
Transductive Multilabel Learning via Label Set Propagation
The problem of multilabel classification has attracted great interest in the last decade, where each instance can be assigned with a set of multiple class labels simultaneously. It has a wide variety of real-world applications, e.g., automatic image ...
Comments
Information & Contributors
Information
Published In
August 2023
481 pages
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 June 2023
Online AM: 22 April 2023
Accepted: 26 February 2023
Revised: 24 February 2023
Received: 22 January 2022
Published in TIST Volume 14, Issue 4
Check for updates
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 172Total Downloads
- Downloads (Last 12 months)55
- Downloads (Last 6 weeks)4
Reflects downloads up to 06 Nov 2024
Other Metrics
Citations
Cited By
View allView Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in