Article

Free access

Self-paced co-training

Authors:

Xuanyi DongAuthors Info & Claims

ICML'17: Proceedings of the 34th International Conference on Machine Learning - Volume 70

Pages 2275 - 2284

Published: 06 August 2017 Publication History

PDF eReader Publisher Site

Abstract

Co-training is a well-known semi-supervised learning approach which trains classifiers on two different views and exchanges labels of unlabeled instances in an iterative way. During co-training process, labels of unlabeled instances in the training pool are very likely to be false especially in the initial training rounds, while the standard co-training algorithm utilizes a "draw without replacement" manner and does not remove these false labeled instances from training. This issue not only tends to degenerate its performance but also hampers its fundamental theory. Besides, there is no optimization model to explain what objective a co-training process optimizes. To these issues, in this study we design a new co-training algorithm named self-paced co-training (SPaCo) with a "draw with replacement" learning mode. The rationality of SPaCo can be proved under theoretical assumptions utilized in traditional co-training research, and furthermore, the algorithm exactly complies with the alternative optimization process for an optimization model of self-paced curriculum learning, which can be finely explained in robust learning manner. Experimental results substantiate the superiority of the proposed method as compared with current state-of-the-art co-training methods.

References

[1]

Abney, Steven. Bootstrapping. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp. 360-367. Association for Computational Linguistics, 2002.

Digital Library

[2]

Balcan, Maria-Florina, Blum, Avrim, and Yang, Ke. Co-training and expansion: Towards bridging theory and practice. In Advances in neural information processing systems, pp. 89-96, 2004.

[3]

Bengio, Yoshua, Louradour, J érôme, Collobert, Ronan, and Weston, Jason. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pp. 41-48. ACM, 2009.

Digital Library

[4]

Blum, Avrim and Mitchell, Tom. Combining labeled and unlabeled data with co-training. In Proceedings of the eleventh annual conference on Computational learning theory, pp. 92-100. ACM, 1998.

Digital Library

[5]

Boyd, Stephen, Parikh, Neal, Chu, Eric, Peleato, Borja, and Eckstein, Jonathan. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends® in Machine Learning, 3(1):1-122, 2011.

Digital Library

[6]

Brefeld, Ulf and Scheffer, Tobias. Co-em support vector learning. In Proceedings of the twenty-first international conference on Machine learning, pp. 16. ACM, 2004.

Digital Library

[7]

Brefeld, Ulf and Scheffer, Tobias. Semi-supervised learning for structured output variables. In Proceedings of the 23rd international conference on Machine learning, pp. 145-152. ACM, 2006.

Digital Library

[8]

Goldman, Sally and Zhou, Yan. Enhancing supervised learning with unlabeled data. In ICML, pp. 327-334, 2000.

Digital Library

[9]

Jiang, Lu, Meng, Deyu, Mitamura, Teruko, and Hauptmann, Alexander G. Easy samples first: Self-paced reranking for zero-example multimedia search. In Proceedings of the 22nd ACM international conference on Multimedia, pp. 547-556. ACM, 2014a.

Digital Library

[10]

Jiang, Lu, Meng, Deyu, Yu, Shoou-I, Lan, Zhenzhong, Shan, Shiguang, and Hauptmann, Alexander. Self-paced learning with diversity. In Advances in Neural Information Processing Systems, pp. 2078-2086, 2014b.

Digital Library

[11]

Jiang, Lu, Meng, Deyu, Zhao, Qian, Shan, Shiguang, and Hauptmann, Alexander G. Self-paced curriculum learning. In AAAI, volume 2, pp. 2694-2700, 2015.

Digital Library

[12]

Kumar, M Pawan, Packer, Benjamin, and Koller, Daphne. Self-paced learning for latent variable models. In Advances in Neural Information Processing Systems, pp. 1189-1197, 2010.

Digital Library

[13]

Li, Ming and Zhou, Zhi-Hua. Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 37(6):1088-1098, 2007.

Digital Library

[14]

Ma, Zilu, Liu, Shiqi, and Meng, Deyu. On convergence property of implicit self-paced objective. CoRR, abs/1703.09923, 2017. URL http://arxiv.org/abs/1703.09923.

[15]

Meng, Deyu and Zhao, Qian. What objective does self-paced learning indeed optimize? In arXiv:1511.06049, 2015.

[16]

Mitchell, Tom. Machine Learning. McGraw Hill, 1997.

Digital Library

[17]

Nigam, Kamal and Ghani, Rayid. Analyzing the effectiveness and applicability of co-training. In Proceedings of the ninth international conference on Information and knowledge management, pp. 86-93. ACM, 2000.

Digital Library

[18]

Pi, Te, Li, Xi, Zhang, Zhongfei, Meng, Deyu, Wu, Fei, Xiao, Jun, and Zhuang, Yueting. Self-paced boost learning for classification. In IJCAI, 2016.

Digital Library

[19]

Scudder, H. Probability of error of some adaptive pattern-recognition machines. IEEE Transactions on Information Theory, 11(3):363-371, 1965.

Digital Library

[20]

Sindhwani, Vikas and Rosenberg, David S. An rkhs for multi-view learning and manifold co-regularization. In Proceedings of the 25th international conference on Machine learning, pp. 976-983. ACM, 2008.

Digital Library

[21]

Sindhwani, Vikas, Niyogi, Partha, and Belkin, Mikhail. A co-regularization approach to semi-supervised learning with multiple views. In Proceedings of ICML workshop on learning with multiple views, pp. 74-79. Citeseer, 2005.

[22]

Supancic, James S and Ramanan, Deva. Self-paced learning for long-term tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 2379-2386, 2013.

Digital Library

[23]

Wang, Wei and Zhou, Zhi-Hua. A new analysis of co-training. In Proceedings of the 27th international conference on machine learning (ICML-10), pp. 1135-1142, 2010.

Digital Library

[24]

Wang, Wei and Zhou, Zhi-Hua. Co-training with insufficient views. In ACML, pp. 467-482, 2013.

[25]

Xu, Qian, Hu, Derek Hao, Xue, Hong, Yu, Weichuan, and Yang, Qiang. Semi-supervised protein subcellular localization. BMC bioinformatics, 10(1):1, 2009.

[26]

Ye, Han-Jia, Zhan, De-Chuan, Miao, Yuan, Jiang, Yuan, and Zhou, Zhi-Hua. Rank consistency based multi-view learning: A privacy-preserving approach. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, pp. 991-1000. ACM, 2015.

Digital Library

[27]

Yu, S., Jiang, L., Mao, Z., Chang, X. J., Du, X. Z., Gan, C., Lan, Z. Z., Xu, Z. W., Li, X. C., Cai, Y., Kumar, A., Miao, Y., Martin, L., Wolfe, N., Xu, S. C., Li, H., Lin, M., Ma, Z. G., Yang, Y., Meng, D. Y., Shan, S. G., Sahin, P. D., Burger, S., Metze, F., Singh, R., Raj, B., Mitamura, T., Stern, R., and Hauptmann, A. CmuinformediaTRECVID 2014. In TRECVID, 2014.

[28]

Zhang, Dingwen, Meng, Deyu, and Han, Junwei. Co-saliency detection via a self-paced multiple-instance learning framework. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015.

Digital Library

[29]

Zhang, Min-Ling and Zhou, Zhi-Hua. Cotrade: confident co-training with data editing. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 41(6):1612-1626, 2011.

Digital Library

[30]

Zhao, Qian, Meng, Deyu, Jiang, Lu, Xie, Qi, Xu, Zongben, and Hauptmann, Alexander G. Self-paced learning for matrix factorization. In AAAI, pp. 3196-3202, 2015.

Digital Library

[31]

Zheng, L., Yang, Y., and Hauptmann, A. G. Person Re-identification: Past, Present and Future. ArXiv e-prints, October 2016.

[32]

Zheng, Liang, Shen, Liyue, Tian, Lu, Wang, Shengjin, Wang, Jingdong, and Tian, Qi. Scalable person re-identification: A benchmark. In Proceedings of the IEEE International Conference on Computer Vision, pp. 1116-1124, 2015.

Digital Library

[33]

Zheng, Zhedong, Zheng, Liang, and Yang, Yi. A discriminatively learned CNN embedding for person re-identification. CoRR, abs/1611.05666, 2016.

Digital Library

[34]

Zhou, Zhi-Hua, Zhan, De-Chuan, and Yang, Qiang. Semi-supervised learning with very few labeled training examples. In Proceedings of the national conference on artificial intelligence, volume 22, pp. 675. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2007.

Digital Library

[35]

Zhu, Xiaojin. Semi-supervised learning. In Encyclopedia of machine learning, pp. 892-897. Springer, 2011.

Cited By

Kim JBaratin AZhang YLacoste-Julien SKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)CrossSplitProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619079(16377-16392)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619079
Fu YLin NYu XJiang S(2023)Self-Training With Double Selectors for Low-Resource Named Entity RecognitionIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2023.325082831(1265-1275)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TASLP.2023.3250828
Huang LLiu YZhou XYou ALi MWang BZhang YPan PYinghui XShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Once and for AllProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3481541(1148-1156)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3481541
Show More Cited By

Self-paced co-training
1. Computing methodologies

Recommendations

Self-paced multi-label co-training
Abstract
Multi-label learning aims to solve classification problems where instances are associated with a set of labels. In reality, it is generally easy to acquire unlabeled data but expensive or time-consuming to label them, and this ...
Self-paced Safe Co-training for Regression
Advances in Knowledge Discovery and Data Mining
Abstract
In semi-supervised learning, co-training is successfully in augmenting the training data with predicted pseudo-labels. With two independently trained regressors, a co-trainer iteratively exchanges their selected instances coupled with pseudo-...
Self-paced multi-view co-training

Co-training is a well-known semi-supervised learning approach which trains classifiers on two or more different views and exchanges pseudo labels of unlabeled instances in an iterative way. During the co-training process, pseudo labels of unlabeled ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ICML'17: Proceedings of the 34th International Conference on Machine Learning - Volume 70

August 2017

4208 pages

Publisher

JMLR.org

Publication History

Published: 06 August 2017

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
286
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)9

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim JBaratin AZhang YLacoste-Julien SKrause ABrunskill ECho KEngelhardt BSabato SScarlett J(2023)CrossSplitProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619079(16377-16392)Online publication date: 23-Jul-2023
https://dl.acm.org/doi/10.5555/3618408.3619079
Fu YLin NYu XJiang S(2023)Self-Training With Double Selectors for Low-Resource Named Entity RecognitionIEEE/ACM Transactions on Audio, Speech and Language Processing10.1109/TASLP.2023.325082831(1265-1275)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TASLP.2023.3250828
Huang LLiu YZhou XYou ALi MWang BZhang YPan PYinghui XShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Once and for AllProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3481541(1148-1156)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3481541
Lv JLiu KHe SShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Differentiated Learning for Multi-Modal Domain AdaptationProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475660(1322-1330)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475660
Mukherjee SAwadallah ALarochelle HRanzato MHadsell RBalcan MLin H(2020)Uncertainty-aware self-training for few-shot text classificationProceedings of the 34th International Conference on Neural Information Processing Systems10.5555/3495724.3497504(21199-21212)Online publication date: 6-Dec-2020
https://dl.acm.org/doi/10.5555/3495724.3497504
Yang LChen ZGu JGuo Y(2019)Dual self-paced graph convolutional networkProceedings of the 28th International Joint Conference on Artificial Intelligence10.5555/3367471.3367606(4062-4069)Online publication date: 10-Aug-2019
https://dl.acm.org/doi/10.5555/3367471.3367606
Fan HZheng LYan CYang Y(2018)Unsupervised Person Re-identificationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/324331614:4(1-18)Online publication date: 10-Oct-2018
https://dl.acm.org/doi/10.1145/3243316

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents