research-article

Prototype vector machine for large scale semi-supervised learning

Authors:

Bahram ParvinAuthors Info & Claims

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Pages 1233 - 1240

https://doi.org/10.1145/1553374.1553531

Published: 14 June 2009 Publication History

Abstract

Practical data mining rarely falls exactly into the supervised learning scenario. Rather, the growing amount of unlabeled data poses a big challenge to large-scale semi-supervised learning (SSL). We note that the computational intensiveness of graph-based SSL arises largely from the manifold or graph regularization, which in turn lead to large models that are difficult to handle. To alleviate this, we proposed the prototype vector machine (PVM), a highly scalable, graph-based algorithm for large-scale SSL. Our key innovation is the use of "prototypes vectors" for efficient approximation on both the graph-based regularizer and model representation. The choice of prototypes are grounded upon two important criteria: they not only perform effective low-rank approximation of the kernel matrix, but also span a model suffering the minimum information loss compared with the complete model. We demonstrate encouraging performance and appealing scaling properties of the PVM on a number of machine learning benchmark data sets.

References

[1]

Bie, T. D., & Cristianini, N. (2004). Convex methods for transduction. Advances in Neural Information Processing Systems 16 (pp. 73--80).

[2]

Collobert, R., Sinz, F., Weston, J., Bottou, L., & Joachims, T. (2006). Large scale transductive svms. Journal of Machine Learning Research, 7, 2006.

Digital Library

[3]

Delalleau, O., Bengio, Y., & Roux, N. (2005). Efficient non-parametric function induction in semi-supervised learning. Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (pp. 96--103).

[4]

Fung, G., & Mangasarian, O. L. (2001). Semi-supervised support vector machines for unlabeled data classification. Optimization Methods and Software, 15, 29--44.

[5]

Goldberger, J., & Roweis, S. (2005). Hierarchical clustering of a mixture model. Advances in Neural Information Processing Systems 17 (pp. 505--512).

[6]

Gustavo, C., Marsheva, T., & Zhou, D. (2007). Semi-supervised graph-based hyperspectral image classification. IEEE Transactions on Geoscience and Remote Sensing, 45, 3044--3054.

[7]

Joachims, T. (1999). Transductive inference for text classification using support vector machines. International Conference on Machine Learning (pp. 200--209). Morgan Kaufmann.

Digital Library

[8]

Lawrence, N., & Jordan, M. (2003). Semi-supervised learning via gaussian processes. Advances in Neural Information Processing Systems 14 (pp. 753--760).

[9]

M. Belkin, M., Niyogi, P., & Sindhwani, V. (2006). Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7, 2399--2434.

Digital Library

[10]

Olivier Chapelle, B. S., & Zien, A. (2006). Semi-supervised learning. MIT.

[11]

Platt, J. (1999). Fast training of support vector machines using sequential minimal optimization. In Advances in kernel methods -- Support vector learning, 185--208.

Digital Library

[12]

Williams, C., & Seeger, M. (2000). The effect of the input density distribution on kernel-based classifiers. Proceedings of the 17th International Conference on Machine Learning (pp. 1159--1166).

Digital Library

[13]

Williams, C., & Seeger, M. (2001). Using the Nyströöm method to speed up kernel machines. Advances in Neural Information Processing Systems 13 (pp. 682--688).

[14]

Xu, Z., Jin, R., Zhu, J., King, I., & Lyu, M. (2008). Efficient convex relaxation for transductive support vector machine. In Advances in neural information processing systems 20, 1641--1648.

[15]

Zhang, K., & Kwok, J. (2008). Improved Nyströöm low rank approximation and error analysis. Proceedings of the 25th international conference on Machine learning (pp. 1232--1239).

Digital Library

[16]

Zhou, D., Bousquet, O., Lal, T. N., Weston, J., & Schöölkopf, B. (2003). Learning with local and global consistency. Neural Information Processing Systems 16 (pp. 321--328).

[17]

Zhu, X., Ghahramani, Z., & Lafferty, J. (2003). Semi-supervised learning using gaussian fields and harmonic functions. In ICML (pp. 912--919).

[18]

Zhu, X., & Lafferty, J. (2005). Harmonic mixtures: combining mixture models and graph-based methods for inductive and scalable semi-supervised learning. The 22nd International Conference on Machine Learning (pp. 1052--1059).

Digital Library

Cited By

Zhou TWang W(2024)Prototype-Based Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338711646:10(6858-6872)Online publication date: Oct-2024
https://doi.org/10.1109/TPAMI.2024.3387116
Qin ZHan CWang QXiushan NYin YLu XOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Unified 3D segmenter as prototypical classifiersProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668135(46419-46432)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668135
Sun YShi ZLi YOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)A graph-theoretic framework for understanding open-world semi-supervised learningProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667162(23934-23967)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667162
Show More Cited By

Recommendations

Semi-supervised learning using randomized mincuts
ICML '04: Proceedings of the twenty-first international conference on Machine learning

In many application domains there is a large amount of unlabeled data but only a very limited amount of labeled training data. One general approach that has been explored for utilizing this unlabeled data is to construct a graph on all the data points ...
Semi-supervised learning using label mean
ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

Semi-Supervised Support Vector Machines (S3VMs) typically directly estimate the label assignments for the unlabeled instances. This is often inefficient even with recent advances in the efficient training of the (supervised) SVM. In this paper, we show ...
Beyond the point cloud: from transductive to semi-supervised learning
ICML '05: Proceedings of the 22nd international conference on Machine learning

Due to its occurrence in engineering domains and implications for natural learning, the problem of utilizing unlabeled data is attracting increasing attention in machine learning. A large body of recent literature has focussed on the transductive ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '09: Proceedings of the 26th Annual International Conference on Machine Learning

June 2009

1331 pages

ISBN:9781605585161

DOI:10.1145/1553374

General Chair:
Andrea Danyluk
Williams College
,
Program Chairs:
Léon Bottou
NEC Laboratories America
,
Michael Littman
Rutgers University

Copyright © 2009 Copyright 2009 by the author(s)/owner(s).

Sponsors

NSF
Microsoft Research: Microsoft Research
MITACS

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

U.S. Department of Energy, Office of Biological and Environmental Research with the University of California
Research Grants Council, University Grants Committee, Hong Kong

Conference

ICML '09

Sponsor:

Microsoft Research

ICML '09: The 26th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

June 14 - 18, 2009

Quebec, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 140 of 548 submissions, 26%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

72
Total Citations
View Citations
585
Total Downloads

Downloads (Last 12 months)40
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Zhou TWang W(2024)Prototype-Based Semantic SegmentationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.338711646:10(6858-6872)Online publication date: Oct-2024
https://doi.org/10.1109/TPAMI.2024.3387116
Qin ZHan CWang QXiushan NYin YLu XOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)Unified 3D segmenter as prototypical classifiersProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668135(46419-46432)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668135
Sun YShi ZLi YOh ANaumann TGloberson ASaenko KHardt MLevine S(2023)A graph-theoretic framework for understanding open-world semi-supervised learningProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3667162(23934-23967)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3667162
Chowdhury ASrinivasan SMukherjee ABhowmick SGhosh K(2023)Improving Node Classification Accuracy of GNN through Input and Output InterventionACM Transactions on Knowledge Discovery from Data10.1145/361053518:1(1-31)Online publication date: 6-Sep-2023
https://dl.acm.org/doi/10.1145/3610535
Gupta ABegum S(2023)An Anchor-Based Fuzzy Rough Feature Selection for Text CategorizationICDSMLA 202110.1007/978-981-19-5936-3_26(281-292)Online publication date: 7-Feb-2023
https://doi.org/10.1007/978-981-19-5936-3_26
Wang ZWang LChan RZeng T(2023)Exploring Latent Sparse Graph for Large-Scale Semi-supervised LearningMachine Learning and Knowledge Discovery in Databases10.1007/978-3-031-26412-2_23(367-383)Online publication date: 17-Mar-2023
https://doi.org/10.1007/978-3-031-26412-2_23
Wang ZZhang LWang RNie FLi X(2022)Semi-supervised Learning via Bipartite Graph Construction with Adaptive NeighborsIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2022.3151315(1-1)Online publication date: 2022
https://doi.org/10.1109/TKDE.2022.3151315
Fu WWang MHao SMu T(2022)FLAG: Faster Learning on Anchor Graph with Label Predictor OptimizationIEEE Transactions on Big Data10.1109/TBDATA.2017.27575228:3(579-591)Online publication date: 1-Jun-2022
https://doi.org/10.1109/TBDATA.2017.2757522
Chen X(2022)Robust Structure-aware Semi-supervised Learning2022 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM54844.2022.00014(41-50)Online publication date: Nov-2022
https://doi.org/10.1109/ICDM54844.2022.00014
Ntakaris AKanniainen JGabbouj MIosifidis A(2020)Mid-price prediction based on machine learning methods with technical and quantitative indicatorsPLOS ONE10.1371/journal.pone.023410715:6(e0234107)Online publication date: 12-Jun-2020
https://doi.org/10.1371/journal.pone.0234107
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents