research-article

Joint learning of labels and distance metric

Authors:

Xian-Sheng HuaAuthors Info & Claims

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, Volume 40, Issue 3

Pages 973 - 978

https://doi.org/10.1109/TSMCB.2009.2034632

Published: 01 June 2010 Publication History

Abstract

Machine learning algorithms frequently suffer from the in sufficiency of training data and the usage of inappropriate distance metric. In this paper, we propose a joint learning of labels and distance metric (JLLDM) approach, which is able to simultaneously address the two difficulties. In comparison with the existing semi-supervised learning and distance metric learning methods that focus only on label prediction or distance metric construction, the JLLDM algorithm optimizes the labels of unlabeled samples and a Mahalanobis distance metric in a unified scheme. The advantage of JLLDM is multifold: 1) the problem of training data insufficiency can be tackled; 2) a good distance metric can be constructed with only very few training samples; and 3) no radius parameter is needed since the algorithm automatically determines the scale of the metric. Extensive experiments are conducted to compare the JLLDM approach with different semi-supervised learning and distance metric learning methods, and empirical results demonstrate its effectiveness.

References

[1]

UCI Repository of Machine Learning Databases. {Online}. Available: http://www.ics.uci.edu/~mlearning/

[2]

A. Bar-Hillel, T. Hertz, N. Shental, and D. Weinshall, "Learning distance functions using equivalence relations," in Proc. Int. Conf. Mach. Learn., 2003, pp. 11-18.

[3]

K. P. Bennett and A. Demiriz, "Semi-supervised support vector machines," in Proc. Adv. Neural Inf. Process., 1999, pp. 368-374.

[4]

A. Blum and T. Mitchell, "Combining labeled and unlabeled data with co-training," in Proc. COLT, 1998, pp. 92-100.

Digital Library

[5]

O. Chapelle, A. Zien, and B. Scholkopf, Semi-Supervised Learning. Cambridge, MA: MIT Press, 2006.

Digital Library

[6]

S. C. Chen and D. Q. Zhang, "Robust image segmentation using FCM with spatial constraints based on new kernel-induced distance metric," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 4, pp. 1907- 1916, Aug. 2004.

Digital Library

[7]

I. Cohen, F. G. Cozman, N. Sebe, M. C. Cirelo, and T. S. Huang, "Semisupervised learning of classifiers: Theory, algorithms, and their application to human-computer interaction," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 12, pp. 1553-1566, Dec. 2004.

Digital Library

[8]

J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, "Information-theoretic metric learning," in Proc. Int. Conf. Mach. Learn., 2007, pp. 209-216.

[9]

P. W. Frey and D. J. Slate, "Letter recognition using Holland-style adaptive classifiers," Mach. Learn., vol. 6, no. 2, pp. 161-182, Mar. 1991.

[10]

A. Globerson and S. Roweis, "Metric learning by collapsing classes," in Proc. Adv. Neural Inf. Process., 2005, pp. 451-458.

[11]

J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, "Neighborhood component analysis," in Proc. Adv. Neural Inf. Process., 2005, pp. 513-520.

[12]

J. He, M. Li, H. J. Zhang, H. Tong, and C. Zhang, "Generalized manifoldranking-based image retrieval," IEEE Trans. Image Process., vol. 15, no. 10, pp. 3170-3177, Oct. 2006.

Digital Library

[13]

T. Hertz, A. Bar-Hillel, and D. Weinshall, "Boosting margin based distance functions for clustering," in Proc. Int. Conf. Mach. Learn., 2004, pp. 393-400.

[14]

S. C. Hoi, W. Liu, and S. F. Chang, "Semi-supervised distance metric learning for collaborative image retrieval," in Proc. IEEE Conf. Comput. Vis. Pattern Recog., 2008, pp. 1-7.

[15]

T. Joachims, "Transductive inference for text classification using support vector machines," in Proc. Int. Conf. Mach. Learn., 1999, pp. 200-209.

Digital Library

[16]

S. Kumar and H. A. Rowley, "Classification of weakly-labeled data with partial equivalence relations," in Proc. Int. Conf. Comput. Vis., 2007, pp. 1-8.

[17]

W. Liu, S. C. H. Hoi, and J. Liu, "Output regularized metric learning with side information," in Proc. 10th Eur. Conf. Comput. Vis., 2008, pp. 358-371.

[18]

D. J. Miller and H. S. Uyar, "A mixture of experts classifier with learning based on both labeled and unlabeled data," in Proc. Adv. Neural Inf. Process., 1997, pp. 571-577.

[19]

K. Nigam, A. K. McCallum, S. Thrun, and T.Mitchell, "Text classification from labeled and unlabeled documents using EM," Mach. Learn., vol. 39, no. 2/3, pp. 103-134, May/Jun. 2000.

Digital Library

[20]

Z. Y. Niu, D. H. Ji, and C. L. Tan, "Word sense disambiguation using label propagation based semi-supervised learning," in Proc. Meeting Assoc. Comput. Linguistics, 2005, pp. 395-402.

[21]

D. Prokhorov, IJCNN 2001 Neural Network Competition. Dearborn, MI: Ford Res. Lab., 2001.

[22]

S. T. Roweis and L. K. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science, vol. 290, no. 5500, pp. 2323-2326, Dec. 2000.

[23]

F. Wang and C. Zhang, "Label propagation through linear neighborhoods," IEEE Trans. Knowl. Data Eng., vol. 20, no. 1, pp. 55-67, Jan. 2008.

Digital Library

[24]

M. Wang, X. S. Hua, R. Hong, J. Tang, G. J. Qi, and Y. Song, "Unified video annotation via multi-graph learning," IEEE Trans. Circuits Syst. Video Technol., vol. 19, no. 5, pp. 733-746, 2009.

Digital Library

[25]

M. Wang, X. S. Hua, J. Tang, and R. Hong, "Beyond distance measurement: Constructing neighborhood similarity for video annotation," IEEE Trans. Multimedia, vol. 11, no. 3, pp. 465-476, Apr. 2009.

Digital Library

[26]

K. Q. Weinberger, J. Blitzer, and L. Saul, "Distance metric learning for large margin nearest neighbor classification," in Proc. Adv. Neural Inf. Process., 2006, pp. 1473-1480.

[27]

E. P. Xing, A. Y. Ng, and S. Russell, "Distance metric learning, with application to clustering with side-information," in Proc. Adv. Neural Inf. Process., 2002, pp. 505-512.

[28]

L. Yang, R. Jin, R. Sukthankar, and Y. Liu, "An efficient algorithm for local distance metric learning," in Proc. AAAI Conf. Artif. Intell., 2006, pp. 543-548.

[29]

D. Y. Yeung, H. Chang, and G. Dai, "A scalable kernel-based algorithm for semi-supervised metric learning," in Proc. Int. Joint Conf. Artif. Intell., 2007, pp. 1138-1143.

[30]

B. Zhao, F. Wang, and C. Zhang, "Smoothness maximization via gradient descents," in Proc. Int. Conf. Acoust., Speech, Signal Process., 2006, pp. II-609-II-612.

[31]

D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Schölkopf, "Learning with local and global consistency," in Proc. Adv. Neural Inf. Process., 2004, pp. 321-328.

[32]

X. Zhu, "Semi-supervised learning literature survey," Univ. Wisconsin-Madison, Madison, WI, Tech. Rep. 1530.

[33]

X. Zhu, "Semi-supervised learning with graphs," Ph.D. dissertation, Carnegie Mellon Univ., Pittsburgh, PA.

[34]

X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-supervised learning using Gaussian fields and harmonic functions," in Proc. Int. Conf. Mach. Learn., 2003, pp. 912-919.

[35]

W. Zuo, D. Zhang, and K. Wang, "Bidirectional PCA with assembled matrix distance metric for image recognition," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 4, pp. 863-872, Aug. 2006.

Digital Library

Cited By

Hu HWang KLv CWu JYang Z(2019)Semi-Supervised Metric Learning-Based Anchor Graph Hashing for Large-Scale Image RetrievalIEEE Transactions on Image Processing10.1109/TIP.2018.286089828:2(739-754)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1109/TIP.2018.2860898
Liu BYuan XYu YLiu QMetaxas D(2017)Parallel Sparse Subspace Clustering via Joint Sample and Parameter Blockwise PartitionACM Transactions on Embedded Computing Systems10.1145/306331616:3(1-17)Online publication date: 9-May-2017
https://dl.acm.org/doi/10.1145/3063316
Li JWu YLu K(2017)Structured Domain AdaptationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2016.253954127:8(1700-1713)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1109/TCSVT.2016.2539541
Show More Cited By

Joint learning of labels and distance metric
1. Computing methodologies

Recommendations

Multi-instance multi-label distance metric learning for genome-wide protein function prediction

Multi-instance multi-label (MIML) learning has been proven to be effective for the genome-wide protein function prediction problems where each training example is associated with not only multiple instances but also multiple class labels. To find an ...
A Robust Distance with Correlated Metric Learning for Multi-Instance Multi-Label Data
MM '16: Proceedings of the 24th ACM international conference on Multimedia

In multi-instance data, every object is a bag that contains multiple elements or instances. Each bag may be assigned to one or more classes, such that it has at least one instance corresponding to every assigned class. However, since the annotations are ...
Semi-supervised Coefficient-Based Distance Metric Learning
Neural Information Processing
Abstract
Distance metric learning plays an important role in real-world applications, such as image classification and clustering. Previous works mainly learn a distance metric through learning a Mahalanobis metric or learning a linear transformation. In ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics

IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics Volume 40, Issue 3

Special issue on game theory

June 2010

425 pages

ISSN:1083-4419

Issue’s Table of Contents

Copyright © 2010.

Publisher

IEEE Press

Publication History

Published: 01 June 2010

Revised: 02 July 2009

Received: 14 January 2009

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Hu HWang KLv CWu JYang Z(2019)Semi-Supervised Metric Learning-Based Anchor Graph Hashing for Large-Scale Image RetrievalIEEE Transactions on Image Processing10.1109/TIP.2018.286089828:2(739-754)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1109/TIP.2018.2860898
Liu BYuan XYu YLiu QMetaxas D(2017)Parallel Sparse Subspace Clustering via Joint Sample and Parameter Blockwise PartitionACM Transactions on Embedded Computing Systems10.1145/306331616:3(1-17)Online publication date: 9-May-2017
https://dl.acm.org/doi/10.1145/3063316
Li JWu YLu K(2017)Structured Domain AdaptationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2016.253954127:8(1700-1713)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1109/TCSVT.2016.2539541
Li CJiang LLi HWu JZhang P(2017)Toward value difference metric with attribute weightingKnowledge and Information Systems10.1007/s10115-016-0960-x50:3(795-825)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1007/s10115-016-0960-x
Liu BYuan XZhang SLiu QMetaxas D(2016)Efficient k-support-norm regularized minimization via fully corrective frank-Wolfe methodProceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence10.5555/3060832.3060867(1760-1766)Online publication date: 9-Jul-2016
https://dl.acm.org/doi/10.5555/3060832.3060867
Dhawane RJoglekar B(2016)A Hybrid Distance Metric Learning for Image RankingProceedings of the Third International Symposium on Computer Vision and the Internet10.1145/2983402.2983442(136-140)Online publication date: 21-Sep-2016
https://dl.acm.org/doi/10.1145/2983402.2983442
Zhuo LGeng ZZhang JLi X(2016)ORB feature based web pornographic image recognitionNeurocomputing10.1016/j.neucom.2015.06.055173:P3(511-517)Online publication date: 15-Jan-2016
https://dl.acm.org/doi/10.1016/j.neucom.2015.06.055
Yu QWang RYang XLi BYao M(2016)Diagonal principal component analysis with non-greedy ℓ1-norm maximization for face recognitionNeurocomputing10.1016/j.neucom.2015.06.011171:C(57-62)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1016/j.neucom.2015.06.011
Xiaozhao Fang Yong Xu Xuelong Li Zhihui Lai Wong W(2015)Learning a Nonnegative Sparse Graph for Linear RegressionIEEE Transactions on Image Processing10.1109/TIP.2015.242554524:9(2760-2771)Online publication date: 1-Sep-2015
https://dl.acm.org/doi/10.1109/TIP.2015.2425545
Yu JGuo YTao DWan J(2015)Human pose recovery by supervised spectral embeddingNeurocomputing10.1016/j.neucom.2015.04.005166:C(301-308)Online publication date: 20-Oct-2015
https://dl.acm.org/doi/10.1016/j.neucom.2015.04.005
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents