research-article

Confidence-weighted linear classification

Authors:

Fernando PereiraAuthors Info & Claims

ICML '08: Proceedings of the 25th international conference on Machine learning

Pages 264 - 271

https://doi.org/10.1145/1390156.1390190

Published: 05 July 2008 Publication History

Abstract

We introduce confidence-weighted linear classifiers, which add parameter confidence information to linear classifiers. Online learners in this setting update both classifier parameters and the estimate of their confidence. The particular online algorithms we study here maintain a Gaussian distribution over parameter vectors and update the mean and covariance of the distribution with each instance. Empirical evaluation on a range of NLP tasks show that our algorithm improves over other state of the art online and batch methods, learns faster in the online setting, and lends itself to better classifier combination after parallel training.

References

[1]

Blitzer, J., Dredze, M., & Pereira, F. (2007). Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. Association of Computational Linguistics (ACL).

[2]

Bordes, A., & Bottou, L. (2005). The huller: a simple and efficient online svm. European Conference on Machine Learning( ECML ), LNAI 3720.

Digital Library

[3]

Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge University Press.

Digital Library

[4]

Carvalho, V. R., & Cohen, W. W. (2006). Single-pass online learning: Performance, voting schemes and online feature selection. KDD-2006.

Digital Library

[5]

Cesa-Bianchi, N., Conconi, A., & Gentile, C. (2005). A second-order perceptron algorithm. SIAM Journal on Computing, 34, 640--668.

Digital Library

[6]

Chang, C.-C., & Lin, C.-J. (2001). LIBSVM: a library for support vector machines. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Digital Library

[7]

Crammer, K., Dekel, O., Keshet, J., Shalev-Shwartz, S., & Singer, Y. (2006). Online passive-aggressive algorithms. JMLR, 7, 551--585.

Digital Library

[8]

Harrington, E., Herbrich, R., Kivinen, J., Platt, J., & Williamson, R. (2003). Online bayes point machines. 7th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD).

Digital Library

[9]

Herbrich, R., Graepel, T., & C. Campbell (2001). Bayes point machinesonline passive-aggressive algorithms. JMLR, 1, 245--279.

Digital Library

[10]

Lewis, D. D., Yand, Y., Rose, T., & Li., F. (2004). Rcv1: A new benchmark collection for text categorization research. JMLR, 5, 361--397.

Digital Library

[11]

McCallum, A. K. (2002). Mallet: A machine learning for language toolkit. http://mallet.cs.umass.edu.

[12]

Petersen, K. B., & Pedersen, M. S. (2007). The matrix cookbook.

[13]

Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. The MIT Press.

Digital Library

[14]

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psych. Rev., 68, 386--407.

[15]

Shivaswamy, P., & Jebara, T. (2007). Ellipsoidal kernel machines. Artificial Intelligence and Statistics.

[16]

Sutton, R. S. (1992). Adapting bias by gradient descent: an incremental version of delta-bar-delta. Proceedings of the Tenth National Conference on Artificial Intelligence (pp. 171--176). MIT Press.

Cited By

Monahov A(2024)Improved Accuracy Metrics for Classification with Imbalanced Data and Where Distance from the Truth Matters, with the Wconf R PackageSSRN Electronic Journal10.2139/ssrn.4802336Online publication date: 2024
https://doi.org/10.2139/ssrn.4802336
Liu YHuang ZXu J(2024)A Space-Efficient One-Pass Online SVM AlgorithmInternational Journal of Computational Geometry & Applications10.1142/S0218195924500043(1-17)Online publication date: 13-Nov-2024
https://doi.org/10.1142/S0218195924500043
Nie XDeng ZHe MFan MTang Z(2024)Online Active Continual Learning for Robotic Lifelong Object RecognitionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.330890035:12(17790-17804)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3308900
Show More Cited By

Index Terms

Confidence-weighted linear classification

Recommendations

Confidence-weighted linear classification for text categorization

Confidence-weighted online learning is a generalization of margin-based learning of linear classifiers in which the margin constraint is replaced by a probabilistic constraint based on a distribution over classifier weights that is updated online as ...
Confidence-weighted linear classification for text categorization

Confidence-weighted online learning is a generalization of margin-based learning of linear classifiers in which the margin constraint is replaced by a probabilistic constraint based on a distribution over classifier weights that is updated online as ...
Chinese text classification by the Naïve Bayes Classifier and the associative classifier with multiple confidence threshold values

Each type of classifier has its own advantages as well as certain shortcomings. In this paper, we take the advantages of the associative classifier and the Naive Bayes Classifier to make up the shortcomings of each other, thus improving the accuracy of ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICML '08: Proceedings of the 25th international conference on Machine learning

July 2008

1310 pages

ISBN:9781605582054

DOI:10.1145/1390156

General Chair:
William Cohen
Carnegie Mellon University
,
Program Chairs:
Andrew McCallum
University of Massachusetts Amherst
,
Sam Roweis
University of Toronto and Google

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Pascal
University of Helsinki
Xerox
Federation of Finnish Learned Societies
Google Inc.
NSF
Machine Learning Journal/Springer
Microsoft Research: Microsoft Research
Intel: Intel
Yahoo!
Helsinki Institute for Information Technology
IBM: IBM

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Defense Advanced Research Projects Agency

Conference

ICML '08

Sponsor:

Microsoft Research
Intel
IBM

ICML '08: The 25th Annual International Conference on Machine Learning held in conjunction with the 2007 International Conference on Inductive Logic Programming

July 5 - 9, 2008

Helsinki, Finland

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

251
Total Citations
View Citations
1,292
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Monahov A(2024)Improved Accuracy Metrics for Classification with Imbalanced Data and Where Distance from the Truth Matters, with the Wconf R PackageSSRN Electronic Journal10.2139/ssrn.4802336Online publication date: 2024
https://doi.org/10.2139/ssrn.4802336
Liu YHuang ZXu J(2024)A Space-Efficient One-Pass Online SVM AlgorithmInternational Journal of Computational Geometry & Applications10.1142/S0218195924500043(1-17)Online publication date: 13-Nov-2024
https://doi.org/10.1142/S0218195924500043
Nie XDeng ZHe MFan MTang Z(2024)Online Active Continual Learning for Robotic Lifelong Object RecognitionIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.330890035:12(17790-17804)Online publication date: Dec-2024
https://doi.org/10.1109/TNNLS.2023.3308900
Liu ZHe X(2024)Dynamic Submodular-Based Learning Strategy in Imbalanced Drifting Streams for Real-Time Safety Assessment in Nonstationary EnvironmentsIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.329478835:3(3038-3051)Online publication date: Mar-2024
https://doi.org/10.1109/TNNLS.2023.3294788
Guo ZWu HFang JZhang JLong J(2024)Online Transfer Learning With Pseudo Label for Gait Phase PredictionIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.348020373(1-15)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3480203
AJIMOTO KYAMAMOTO YKUSUNOKI YNAKASHIMA T(2024)A Study on Multi-Class Online Fuzzy Classifiers for Dynamic Environments2024 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)10.1109/FUZZ-IEEE60900.2024.10612027(1-7)Online publication date: 30-Jun-2024
https://doi.org/10.1109/FUZZ-IEEE60900.2024.10612027
Anwar NBattula MKaur JPailwan S(2024)Learning Cost-Adjusted Predictive Models with Margin-Based Framework2024 IEEE 14th Annual Computing and Communication Workshop and Conference (CCWC)10.1109/CCWC60891.2024.10427738(0708-0712)Online publication date: 8-Jan-2024
https://doi.org/10.1109/CCWC60891.2024.10427738
Khoei TSingh A(2024)Data reduction in big data: a survey of methods, challenges and future directionsInternational Journal of Data Science and Analytics10.1007/s41060-024-00603-zOnline publication date: 10-Jul-2024
https://doi.org/10.1007/s41060-024-00603-z
Wu JLin YLu HHang H(2024)Online Linearized Confidence‐Weighted Learning on a BudgetAdvanced Intelligent Systems10.1002/aisy.202400345Online publication date: 17-Nov-2024
https://doi.org/10.1002/aisy.202400345
Cai YLi XLi J(2023)Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive ReviewSensors10.3390/s2305245523:5(2455)Online publication date: 23-Feb-2023
https://doi.org/10.3390/s23052455
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents