Article

k-means++: the advantages of careful seeding

Authors:

Sergei VassilvitskiiAuthors Info & Claims

SODA '07: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms

Pages 1027 - 1035

Published: 07 January 2007 Publication History

Abstract

The k-means method is a widely used clustering technique that seeks to minimize the average squared distance between points in the same cluster. Although it offers no accuracy guarantees, its simplicity and speed are very appealing in practice. By augmenting k-means with a very simple, randomized seeding technique, we obtain an algorithm that is Θ(logk)-competitive with the optimal clustering. Preliminary experiments show that our augmentation improves both the speed and the accuracy of k-means, often quite dramatically.

References

[1]

Pankaj K. Agarwal and Nabil H. Mustafa. k-means projective clustering. In PODS '04: Proceedings of the twenty-third ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, pages 155--165, New York, NY, USA, 2004. ACM Press.

Digital Library

[2]

D. Arthur and S. Vassilvitskii. Worst-case and smoothed analysis of the ICP algorithm, with an application to the k-means method. In Symposium on Foundations of Computer Science, 2006.

Digital Library

[3]

David Arthur and Sergei Vassilvitskii. k-means++ test code. http://www.stanford.edu/~darthur/kMeansppTest.zip.

[4]

David Arthur and Sergei Vassilvitskii. How slow is the k-means method? In SCG '06: Proceedings of the twenty-second annual symposium on computational geometry. ACM Press, 2006.

Digital Library

[5]

Pavel Berkhin. Survey of clustering data mining techniques. Technical report, Accrue Software, San Jose, CA, 2002.

[6]

Moses Charikar, Liadan O'Callaghan, and Rina Panigrahy. Better streaming algorithms for clustering problems. In STOC '03: Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, pages 30--39, New York, NY, USA, 2003. ACM Press.

Digital Library

[7]

Philippe Collard's cloud cover database. ftp://ftp.ics.uci.edu/pub/machine-learning-databases/undocumented/taylor/cloud.data.

[8]

Sanjoy Dasgupta. How fast is k-means? In Bernhard Schölkopf and Manfred K. Warmuth, editors, COLT, volume 2777 of Lecture Notes in Computer Science, page 735. Springer, 2003.

[9]

W. Fernandez de la Vega, Marek Karpinski, Claire Kenyon, and Yuval Rabani. Approximation schemes for clustering problems. In STOC '03: Proceedings of the thirty-fifth annual ACM symposium on Theory of computing, pages 50--58, New York, NY, USA, 2003. ACM Press.

Digital Library

[10]

P. Drineas, A. Frieze, R. Kannan, S. Vempala, and V. Vinay. Clustering large graphs via the singular value decomposition. Mach. Learn., 56(1-3):9--33, 2004.

Digital Library

[11]

Frédéric Gibou and Ronald Fedkiw. A fast hybrid k-means level set algorithm for segmentation. In 4th Annual Hawaii International Conference on Statistics and Mathematics, pages 281--291, 2005.

[12]

Sudipto Guha, Adam Meyerson, Nina Mishra, Rajeev Motwani, and Liadan O'Callaghan. Clustering data streams: Theory and practice. IEEE Transactions on Knowledge and Data Engineering, 15(3):515--528, 2003.

Digital Library

[13]

Sariel Har-Peled and Soham Mazumdar. On coresets for k-means and k-median clustering. In STOC '04: Proceedings of the thirty-sixth annual ACM symposium on Theory of computing, pages 291--300, New York, NY, USA, 2004. ACM Press.

Digital Library

[14]

Sariel Har-Peled and Bardia Sadri. How fast is the k-means method? In SODA '05: Proceedings of the sixteenth annual ACM-SIAM symposium on Discrete algorithms, pages 877--885, Philadelphia, PA, USA, 2005. Society for Industrial and Applied Mathematics.

Digital Library

[15]

R. Herwig, A. J. Poustka, C. Muller, C. Bull, H. Lehrach, and J O'Brien. Large-scale clustering of cdna-fingerprinting data. Genome Research, 9:1093--1105, 1999.

[16]

Mary Inaba, Naoki Katoh, and Hiroshi Imai. Applications of weighted voronoi diagrams and randomization to variance-based k-clustering: (extended abstract). In SCG '94: Proceedings of the tenth annual symposium on Computational geometry, pages 332--339, New York, NY, USA, 1994. ACM Press.

Digital Library

[17]

Tapas Kanungo, David M. Mount, Nathan S. Netanyahu, Christine D. Piatko, Ruth Silverman, and Angela Y. Wu. A local search approximation algorithm for k-means clustering. Comput. Geom., 28(2-3):89--112, 2004.

Digital Library

[18]

KDD Cup 1999 dataset. http://kdd.ics.uci.edu//databases/kddcup99/kddcup99.html.

[19]

Amit Kumar, Yogish Sabharwal, and Sandeep Sen. A simple linear time (1 + ε)-approximation algorithm for k-means clustering in any dimensions. In FOCS '04: Proceedings of the 45th Annual IEEE Symposium on Foundations of Computer Science (FOCS'04), pages 454--462, Washington, DC, USA, 2004. IEEE Computer Society.

Digital Library

[20]

Stuart P. Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28(2):129--136, 1982.

[21]

Jiri Matousek. On approximate geometric k-clustering. Discrete & Computational Geometry, 24(1):61--84, 2000.

[22]

Ramgopal R. Mettu and C. Greg Plaxton. Optimal time bounds for approximate clustering. In Adnan Darwiche and Nir Friedman, editors, UAI, pages 344--351. Morgan Kaufmann, 2002.

Digital Library

[23]

A. Meyerson. Online facility location. In FOCS '01: Proceedings of the 42nd IEEE symposium on Foundations of Computer Science, page 426, Washington, DC, USA, 2001. IEEE Computer Society.

Digital Library

[24]

R. Ostrovsky, Y. Rabani, L. Schulman, and C. Swamy. The effectiveness of Lloyd-type methods for the k-Means problem. In Symposium on Foundations of Computer Science, 2006.

Digital Library

[25]

Spam e-mail database. http://www.ics.uci.edu/~mlearn/databases/spambase/.

Cited By

Li SDai YBalzarotti DXu W(2024)BackdoorIndicatorProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699135(4193-4210)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699135
Li HPing Y(2024)Selecting Indispensable Edge Patterns With Adaptive Sampling and Double Local Analysis for Data DescriptionJournal of Cases on Information Technology10.4018/JCIT.33594526:1(1-26)Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.4018/JCIT.335945
Yang QDing HLarson K(2024)Approximate algorithms for k-sparse Wasserstein Barycenter with outliersProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/588(5316-5325)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/588
Show More Cited By

k-means++: the advantages of careful seeding
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning

Recommendations

Hybrid Bisect K-Means Clustering Algorithm
BCGIN '11: Proceedings of the 2011 International Conference on Business Computing and Global Informatization

In this paper, we present a hybrid clustering algorithm that combines divisive and agglomerative hierarchical clustering algorithm. Our method uses bisect K-means for divisive clustering algorithm and Unweighted Pair Group Method with Arithmetic Mean (...
DIC-DOC-K-means: Dissimilarity-based Initial Centroid selection for DOCument clustering using K-means for improving the effectiveness of text document clustering

In this article, a new initial centroid selection for a K-means document clustering algorithm, namely, Dissimilarity-based Initial Centroid selection for DOCument clustering using K-means (DIC-DOC-K-means), to improve the performance of text document ...
DK-means: a deterministic K-means clustering algorithm for gene expression analysis

Clustering has been widely applied in interpreting the underlying patterns in microarray gene expression profiles, and many clustering algorithms have been devised for the same. K-means is one of the popular algorithms for gene data clustering due to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SODA '07: Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms

January 2007

1322 pages

ISBN:9780898716245

Conference Chair:
Harold Gabow
University of Colorado, Boulder

Sponsors

SIAM Activity Group on Discrete Mathematics
SIGACT: ACM Special Interest Group on Algorithms and Computation Theory

Publisher

Society for Industrial and Applied Mathematics

United States

Publication History

Published: 07 January 2007

Check for updates

Qualifiers

Article

Acceptance Rates

SODA '07 Paper Acceptance Rate 139 of 382 submissions, 36%;

Overall Acceptance Rate 411 of 1,322 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

620
Total Citations
View Citations
9,132
Total Downloads

Downloads (Last 12 months)570
Downloads (Last 6 weeks)60

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li SDai YBalzarotti DXu W(2024)BackdoorIndicatorProceedings of the 33rd USENIX Conference on Security Symposium10.5555/3698900.3699135(4193-4210)Online publication date: 14-Aug-2024
https://dl.acm.org/doi/10.5555/3698900.3699135
Li HPing Y(2024)Selecting Indispensable Edge Patterns With Adaptive Sampling and Double Local Analysis for Data DescriptionJournal of Cases on Information Technology10.4018/JCIT.33594526:1(1-26)Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.4018/JCIT.335945
Yang QDing HLarson K(2024)Approximate algorithms for k-sparse Wasserstein Barycenter with outliersProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/588(5316-5325)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/588
Peng ZWang ELiu XCheng MLarson K(2024)Let's start overProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/532(4815-4823)Online publication date: 3-Aug-2024
https://dl.acm.org/doi/10.24963/ijcai.2024/532
Hao GHuang HWang HGao JWang HWooldridge MDy JNatarajan S(2024)Composite active learningProceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v38i11.29119(12286-12294)Online publication date: 20-Feb-2024
https://dl.acm.org/doi/10.1609/aaai.v38i11.29119
Solanki BNair ASingha MMukhopadhyay SJha ABanerjee B(2024)GraphVL: Graph-Enhanced Semantic Modeling via Vision-Language Models for Generalized Class Discovery✱Proceedings of the Fifteenth Indian Conference on Computer Vision Graphics and Image Processing10.1145/3702250.3702266(1-10)Online publication date: 13-Dec-2024
https://dl.acm.org/doi/10.1145/3702250.3702266
Wang HBian LXiong HJin H(2024)Fast Candidate Screening for Post-diagnosis RefinementACM Transactions on Design Automation of Electronic Systems10.1145/369819730:1(1-21)Online publication date: 30-Sep-2024
https://dl.acm.org/doi/10.1145/3698197
Bruch SNardini FIngber ALiberty E(2024)Bridging Dense and Sparse Maximum Inner Product SearchACM Transactions on Information Systems10.1145/366532442:6(1-38)Online publication date: 19-Aug-2024
https://dl.acm.org/doi/10.1145/3665324
Draganov ASaulpic DSchwiegelshohn C(2024)Settling Time vs. Accuracy Tradeoffs for Clustering Big DataProceedings of the ACM on Management of Data10.1145/36549762:3(1-25)Online publication date: 30-May-2024
https://dl.acm.org/doi/10.1145/3654976
Kahng AKundu SThumathy S(2024)Scalable Flip-Flop Clustering Using Divide and Conquer For Capacitated K-MeansProceedings of the Great Lakes Symposium on VLSI 202410.1145/3649476.3658744(177-184)Online publication date: 12-Jun-2024
https://dl.acm.org/doi/10.1145/3649476.3658744
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents