A Dual Perspective of Sparse and Robust Online Learning Algorithm
Authors: 
Min Tang, Boliang Sun, Guohui LiAuthors Info & Claims
Min Tang, Boliang Sun, Guohui LiAuthors Info & ClaimsPages 217 - 221
Abstract
In this paper, we propose a dual perspective of online learning algorithm, which concerns using a window method to achieve sparsity and robustness. It makes use of Fenchel conjugates and gradient ascent to perform online learning optimization process. The window method is an update strategy for the classifier. It consists of two bounds which related to the dual increase. The lower bound abandons the points which induce a smaller dual ascent while the upper bound constraints the dual increase generated by noise points to reduce their influence on the target boundary. Moreover, with the use of the window method, the prediction accuracy can be increased significantly. Detailed experiments on artificial and real world datasets verify the utility of the proposed approaches.
References
[1]
K. Crammer, O. Dekel, J. Keshet, et al. Online passive-aggressive algorithms. The Journal of Machine Learning Research, 2006, 7: 551--585.
[2]
J. Kivinen, A. J. Smola, R. C. Williamson. Online learning with kernels. Signal Processing, IEEE Transactions on, 2004, 52(8): 2165--2176.
[3]
A. Blum. On-line algorithms in machine learning. Springer Berlin Heidelberg, 1998.
[4]
N. Cesa-Bianchi, A. Conconi, C. Gentile. On the generalization ability of on-line learning algorithms. Information Theory, IEEE Transactions on, 2004, 50(9): 2050--2057.
[5]
N. Littlestone, M K Warmuth. The weighted majority algorithm. Foundations of Computer Science, 1989., 30th Annual Symposium on. IEEE, 1989: 256--261.
[6]
F. Rosenblatt. The perceptron: a probabilistic model for information storage and organization in the brain. Psychological review, 1958, 65(6): 386
[7]
O. Dekel, S. Shalev-Shwartz, Y. Singer. The Forgetron: A kernel-based perceptron on a fixed budget. 2005.
[8]
J. Langford, L. Li, T. Zhang. Sparse online learning via truncated gradient. The Journal of Machine Learning Research, 2009, 10: 777--801.
[9]
A. B. Goldberg, M. Li, X. Zhu. Online manifold regularization: A new learning setting and empirical study. Machine Learning and Knowledge Discovery in Databases. Springer Berlin Heidelberg, 2008: 393--407.
[10]
M. Zinkevich. Online convex programming and generalized infinitesimal gradient ascent. 2003.
[11]
S. Shalev-Shwartz, Y. Singer. A primal-dual perspective of online learning algorithms. Machine Learning, 2007, 69(2-3): 115--142.
[12]
B. Sun, G. Li, L. Jia, et al. Online manifold regularization by dual ascending procedure. Mathematical Problems in Engineering, 2013, 2013.
[13]
Belkin M, Niyogi P, Sindhwani V. Manifold Regularization, 2004. http://manifold.cs.uchicago.edu/manifold_regularization/data.html.
[14]
Bache, K. & Lichman, M. (2013). UCI Machine Learning Repository {http://archive.ics.uci.edu/ml}. Irvine, CA: University of California, School of Information and Computer Science.
Index Terms
- A Dual Perspective of Sparse and Robust Online Learning Algorithm
Recommendations
A primal-dual perspective of online learning algorithms
We describe a novel framework for the design and analysis of online learning algorithms based on the notion of duality in constrained optimization. We cast a sub-family of universal online bounds as an optimization problem. Using the weak duality ...
Sparse Online Co-regularization Using Conjugate Functions
ICPR '14: Proceedings of the 2014 22nd International Conference on Pattern RecognitionIn this paper, we propose a novel sparse online co-regularization framework for multiview semi-supervised learning, which concerns using a small portion of the arrived training examples to represent predictors during the online learning process. This ...
Sparse Passive-Aggressive Learning for Bounded Online Kernel Methods
Research Survey and Regular PapersOne critical deficiency of traditional online kernel learning methods is their unbounded and growing number of support vectors in the online learning process, making them inefficient and non-scalable for large-scale applications. Recent studies on ...
Comments
Information & Contributors
Information
Published In
July 2014
430 pages
ISBN:9781450328104
DOI:10.1145/2632856
- General Chairs:
- Hanzi Wang,
- Larry Davis,
- Program Chairs:
- Wenwu Zhu,
- Stephan Kopf,
- Yanyun Qu,
- Publications Chairs:
- Jun Yu,
- Jitao Sang,
- Tao Mei
Copyright © 2014 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
In-Cooperation
- NSF of China: National Natural Science Foundation of China
- Beijing ACM SIGMM Chapter
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 10 July 2014
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICIMCS '14
ICIMCS '14: International Conference on Internet Multimedia Computing and Service
July 10 - 12, 2014
Xiamen, China
Acceptance Rates
Overall Acceptance Rate 163 of 456 submissions, 36%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 55Total Downloads
- Downloads (Last 12 months)3
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in