research-article

Kernel-based MinMax clustering methods with kernelization of the metric and auto-tuning hyper-parameters

Authors:

Youyun XuAuthors Info & Claims

Volume 359, Issue C

Pages 173 - 184

https://doi.org/10.1016/j.neucom.2019.05.056

Published: 24 September 2019 Publication History

Abstract

This paper proposes kernel-based MinMax clustering methods with kernelization of the metric and auto-tuning hyper-parameters which learn the variable weights and adjust the cluster weights automatically. We develop the new objective functions that are obtained from the proposed algorithms to achieve the desirable partition by minimizing the dissimilarity measures with kernelization of the metric. Correspondingly, two additional steps are introduced to k-means algorithms, so that, not only the performance is improved, but also the efficiency remains. More specifically, the proposed algorithms learn two types of weights at each iteration where variable weights identify relevant variables and cluster weights to confine the occurrence of the large variance cluster. Finally, the experiments on ten UCI benchmark datasets corroborate the superiority of the proposed algorithms.

References

[1]

P. E. Green, Kim J., F. Carmone, A preliminary study of optimal variable weighting in k-means clustering, J. Classif. 7 (1990) 271–285,.

[2]

Xu R., D. Wunsch, Survey of clustering algorithms, IEEE Trans. Neural Netw. 16 (3) (2005) 645–678,.

Digital Library

[3]

Zhang H., Chow T.W.S., Wu Q.M.J., Organizing books and authors by multilayer SOM, IEEE Trans. Neural Netw. Learn. Syst. 27 (12) (2016) 2537–2550,.

[4]

Zhang H., Wang S., Xu X., Chow T.W.S., Wu Q.M.J., Tree2Vector: learning a vectorial representation for tree-structured data, IEEE Trans. Neural Netw. Learn. Syst. 29 (11) (2018) 5304–5318,.

[5]

A.K. Jain, R.P.W. Duin, Statistical pattern recognition: a review, IEEE Trans. Pattern Anal. Mach. Intell. 22 (1) (2000) 4–37,.

Digital Library

[6]

J. Peña, J. Lozano, P. Larrañaga, An empirical comparison of four initialization methods for the k-means algorithm, Pattern Recognit. Lett. 20 (10) (1999) 1027–1040,.

Digital Library

[7]

M.E. Celebi, H.A. Kingravi, P.A. Vela, A comparative study of efficient initialization methods for the k-means clustering algorithm, Expert Syst. Appl. 40 (1) (2013) 200–210,.

Digital Library

[8]

Huang J.Z., Ng M.K., Rong H., Li Z., Automated variable weighting in k-means type clustering, IEEE Trans. Pattern Anal. Mach. Intell. 27 (5) (2005) 657–668,.

Digital Library

[9]

R.C. de Amorim, B. Mirkin, Minkowski metric, feature weighting and anomalous cluster initializing in k-means clustering, Pattern Recognit. 45 (3) (2012) 1061–1075,.

Digital Library

[10]

Jing L., Ng M.K., Huang J.Z., An entropy weighting k-means algorithm for subspace clustering of high-dimensional sparse data, IEEE Trans. Knowl. Data Eng. 19 (8) (2007) 1026–1041,.

Digital Library

[11]

Tsai C.-Y., Chiu C.-C., Developing a feature weight self-adjustment mechanism for a k-means clustering algorithm, Comput. Stat. Data Anal. 52 (10) (2008) 4658–4672,.

Digital Library

[12]

C. Domeniconi, D. Gunopulos, Ma S., Yan B., M. Al-Razgan, D. Papadopoulos, Locally adaptive metrics for clustering high dimensional data, Data Min. Knowl. Discov. 14 (1) (2007) 63–97,.

Digital Library

[13]

Chen X., Xu X., Huang J.Z., Ye Y., TW-k-Means: automated two-level variable weighting clustering algorithm for multiview data, IEEE Trans. Knowl. Data Eng. 25 (4) (2013) 932–944,.

Digital Library

[14]

Chan E.Y., Ching W.K., Ng M.K., Huang J.Z., An optimization algorithm for clustering using weighted dissimilarity measures, Pattern Recognit. 37 (5) (2004) 943–952,.

[15]

B. Schölkopf, A.J. Smola, K.-R. Müller, Nonlinear component analysis as a kernel eigenvalue problem, Neural Comput. 10 (1998) 1299–1319,.

Digital Library

[16]

M. Girolami, Mercer kernel-based clustering in feature space, IEEE Trans. Neural Netw. 13 (3) (2002) 780–784,.

Digital Library

[17]

M.R. Ferreira, F. de A.T. de Carvalho, Kernel-based hard clustering methods in the feature space with automatic variable weighting, Pattern Recognit. 47 (9) (2014) 3082–3095,.

[18]

M.R. Ferreira, F. de A.T. de Carvalho, Kernel fuzzy c-means with automatic variable weighting, Fuzzy Sets Syst. 237 (2014) 1–46,., Theme: Data Analysis and Fuzzy Models.

Digital Library

[19]

D. Graves, W. Pedrycz, Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study, Fuzzy Sets Syst. 161 (4) (2010) 522–543,., Theme: Forecasting, Classification, and Learning.

Digital Library

[20]

T. Kohonen, The self-organizing map, Proc. IEEE 78 (9) (1990) 1464–1480,.

[21]

Ding Y., Fu X., Kernel-based fuzzy c-means clustering algorithm based on genetic algorithm, Neurocomputing 188 (2016) 233–238,., Advanced Intelligent Computing Methodologies and Applications.

[22]

Zhou J., Chen L., Chen C.P., Zhang Y., Li H.-X., Fuzzy clustering with the entropy of attribute weights, Neurocomputing 198 (2016) 125–134,., Advances in Neural Networks, Intelligent Control and Information Processing.

Digital Library

[23]

M.R. Ferreira, F. de A.T. de Carvalho, E.C. Simões, Kernel-based hard clustering methods with kernelization of the metric and automatic weighting of the variables, Pattern Recognit. 51 (2016) 310–321,.

Digital Library

[24]

M. Filippone, F. Camastra, F. Masulli, S. Rovetta, A survey of kernel and spectral methods for clustering, Pattern Recognit. 41 (1) (2008) 176–190,.

Digital Library

[25]

F. de A.T. de Carvalho, E.C. Simes, L.V. Santana, M.R. Ferreira, Gaussian kernel c-means hard clustering algorithms with automated computation of the width hyper-parameters, Pattern Recognit. 79 (2018) 370–386,.

[26]

Yin J.-B., Li T., Shen H.-B., Gaussian kernel optimization: complex problem and a simple solution, Neurocomputing 74 (18) (2011) 3816–3822,.

Digital Library

[27]

Chen B., Liang J., Zheng N., J.C. Príncipe, Kernel least mean square with adaptive kernel size, Neurocomputing 191 (2016) 95–106,.

Digital Library

[28]

S. Kitayama, K. Yamazaki, Simple estimate of the width in gaussian kernel with adaptive scaling technique, Appl. Soft Comput. 11 (8) (2011) 4726–4737,.

Digital Library

[29]

A.M. Bagirov, J. Ugon, D. Webb, Fast modified global k-means algorithm for incremental cluster construction, Pattern Recognit. 44 (4) (2011) 866–876,.

Digital Library

[30]

Li X., Wong H.-S., Wu S., A fuzzy minimax clustering model and its applications, Inf. Sci. 186 (1) (2012) 114–125,.

Digital Library

[31]

A. Likas, N. Vlassis, J.J. Verbeek, The global k-means clustering algorithm, Pattern Recognit. 36 (2) (2003) 451–461,., Biometrics.

[32]

G. Tzortzis, A. Likas, The minmax k-means clustering algorithm, Pattern Recognit. 47 (7) (2014) 2505–2516,.

[33]

Huang X., Ye Y., Zhang H., Extensions of kmeans-type algorithms: a new clustering framework by integrating intracluster compactness and intercluster separation, IEEE Trans. Neural Netw. Learn. Syst. 25 (8) (2014) 1433–1446,.

[34]

A. Asuncion, D.J. Newman, UCI Machine Learning Repository, University of California, Irvine, 2007.

[35]

B. Caputo, Sim K., F. Furesj, A. Smola, Appearance-based object recognition using SVMs: which kernel should I use, Proceedings of NIPS Workshop on Statistical Methods for Computational Experiments in Visual Processingand Computer Vision, 2002.

[36]

L. Hubert, P. Arabie, Comparing partitions, J. Classif. 2 (1985) 193–218,.

[37]

R. Baeza-Yates, B. Ribeiro-Neto, Modern Information Retrieval, ACM Press, New York, 1999.

Digital Library

[38]

A. Strehl, J. Ghosh, Cluster ensembles - a knowledge reuse framework for combining multiple partitions, J. Mach. Learn. Res. 3 (2002) 583–617,.

Digital Library

[39]

Chiang M., B. Mirkin, Intelligent choice of the number of clusters in k-means clustering: an experimental study with different cluster spreads, J. Classif. 27 (2010) 3–40,.

Digital Library

Index Terms

Kernel-based MinMax clustering methods with kernelization of the metric and auto-tuning hyper-parameters
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
        Cluster analysis
2. Information systems
  1. Information systems applications
    1. Data mining
      1. Clustering

Index terms have been assigned to the content through auto-classification.

Recommendations

Kernel-based hard clustering methods with kernelization of the metric and automatic weighting of the variables

This paper presents kernel-based hard clustering methods with kernelization of the metric and automatic weighting of the variables. The proposed methodology is supported by the fact that a kernel function can be written as a sum of kernels evaluated ...
Kernel-based multiobjective clustering algorithm with automatic attribute weighting

Clustering algorithms with attribute weighting have gained much attention during the last decade. However, they usually optimize a single-objective function that can be a limitation to cope with different kinds of data, especially those with non-hyper-...
A survey of kernel and spectral methods for clustering

Clustering algorithms are a useful tool to explore data structures and have been employed in many disciplines. The focus of this paper is the partitioning clustering problem with a special interest in two recent approaches: kernel and spectral methods. ...

Comments

Information & Contributors

Information

Published In

cover image Neurocomputing

Neurocomputing Volume 359, Issue C

Sep 2019

528 pages

ISSN:0925-2312

Issue’s Table of Contents

Elsevier B.V.

Publisher

Elsevier Science Publishers B. V.

Netherlands

Publication History

Published: 24 September 2019

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents