Online Drift Detection with Maximum Concept Discrepancy
Authors: 
Ke Wan, Yi Liang, Susik YoonAuthors Info & Claims
Ke Wan, Yi Liang, Susik YoonAuthors Info & ClaimsPages 2924 - 2935
Abstract
Continuous learning from an immense volume of data streams becomes exceptionally critical in the internet era. However, data streams often do not conform to the same distribution over time, leading to a phenomenon called concept drift. Since a fixed static model is unreliable for inferring concept-drifted data streams, establishing an adaptive mechanism for detecting concept drift is crucial. Current methods for concept drift detection primarily assume that the labels or error rates of downstream models are given and/or underlying statistical properties exist in data streams. These approaches, however, struggle to address high-dimensional data streams with intricate irregular distribution shifts, which are more prevalent in real-world scenarios. In this paper, we propose MCD-DD, a novel concept drift detection method based on maximum concept discrepancy, inspired by the maximum mean discrepancy. Our method can adaptively identify varying forms of concept drift by contrastive learning of concept embeddings without relying on labels or statistical properties. With thorough experiments under synthetic and real-world scenarios, we demonstrate that the proposed method outperforms existing baselines in identifying concept drifts and enables qualitative analysis with high explainability.
References
[1]
S. Agrahari and A. K. Singh. Concept drift detection in data stream mining: A literature review. Journal of King Saud University-Computer and Information Sciences, 34(10):9523--9540, 2022.
[2]
F. Bayram, B. S. Ahmed, and A. Kassler. From concept drift to model degradation: An overview on performance-aware drift detectors. Knowledge-Based Systems, 245:108632, 2022.
[3]
A. Berlinet and C. Thomas-Agnan. Reproducing kernel Hilbert spaces in probability and statistics. Springer Science & Business Media, 2011.
[4]
A. Bifet and R. Gavalda. Learning from time-changing data with adaptive windowing. In SDM, pages 443--448. SIAM, 2007.
[5]
A. Bifet, R. Gavalda, G. Holmes, and B. Pfahringer. Machine learning for data streams: with practical examples in MOA. MIT press, 2023.
[6]
L. Bu, C. Alippi, and D. Zhao. A pdf-free change detection test based on density difference estimation. IEEE TNNLS, 29(2):324--334, 2018.
[7]
V. Cerqueira, H. M. Gomes, A. Bifet, and L. Torgo. STUDD: A student--teacher method for unsupervised concept drift detection. Machine Learning, 112(11):4351--4378, 2023.
[8]
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton. A simple framework for contrastive learning of visual representations. In ICML, pages 1597--1607. PMLR, 2020.
[9]
D. Chicco and G. Jurman. The advantages of the matthews correlation coefficient (MCC) over f1 score and accuracy in binary classification evaluation. BMC genomics, 21(1):1--13, 2020.
[10]
G. K. Dziugaite, D. M. Roy, and Z. Ghahramani. Training generative neural networks via maximum mean discrepancy optimization. In UAI, pages 258--267, 2015.
[11]
A. Elisseeff and J. Weston. A kernel method for multi-labelled classification. NeurIPS, 14, 2001.
[12]
I. Frias-Blanco, J. del Campo-Ávila, G. Ramos-Jimenez, R. Morales-Bueno, A. Ortiz-Díaz, and Y. Caballero-Mota. Online and non-parametric drift detection methods based on hoeffding's bounds. IEEE TKDE, 27(3):810--823, 2014.
[13]
J. Gama, P. Medas, G. Castillo, and P. Rodrigues. Learning with drift detection. In Advances in Artificial Intelligence--SBIA 2004, pages 286--295. Springer, 2004.
[14]
J. Gama, R. Sebastião, and P. P. Rodrigues. On evaluating stream learning algorithms. Machine Learning, 90(3):317--346, 2013.
[15]
J. Gama, I. ?liobaite, A. Bifet, M. Pechenizkiy, and A. Bouchachia. A survey on concept drift adaptation. CSUR, 46(4):1--37, 2014.
[16]
R. N. Gemaque, A. F. J. Costa, R. Giusti, and E. M. Dos Santos. An overview of unsupervised drift detection methods. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(6):e1381, 2020.
[17]
H. Gouk, E. Frank, B. Pfahringer, and M. J. Cree. Regularisation of neural networks by enforcing lipschitz continuity. Machine Learning, 110:393--416, 2021.
[18]
A. Gretton, K. Borgwardt, M. Rasch, B. Schölkopf, and A. Smola. A kernel method for the two-sample-problem. NeurIPS, 19, 2006.
[19]
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Schölkopf, and A. Smola. A kernel two-sample test. Journal of Machine Learning Research, 13(25):723--773, 2012.
[20]
A. Gretton, K. Fukumizu, Z. Harchaoui, and B. K. Sriperumbudur. A fast, consistent kernel two-sample test. NeurIPS, 22, 2009.
[21]
I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville. Improved training of wasserstein gans. NeurIPS, 30, 2017.
[22]
K. He, H. Fan, Y.Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In CVPR, pages 9729--9738, 2020.
[23]
T. Hofmann, B. Schölkopf, and A. J. Smola. Kernel methods in machine learning. 2008.
[24]
D. Kifer, S. Ben-David, and J. Gehrke. Detecting change in data streams. In VLDB, volume 4, pages 180--191. Toronto, Canada, 2004.
[25]
D. Kim, H. Min, Y. Nam, H. Song, S. Yoon, M. Kim, and J.-G. Lee. Covid-EENet: Predicting fine-grained impact of covid-19 on local economies. In AAAI, volume 36, pages 11971--11981, 2022.
[26]
L. Korycki and B. Krawczyk. Concept drift detection from multi-class imbalanced data streams. In ICDE, pages 1068--1079. IEEE, 2021.
[27]
C.-L. Li, W.-C. Chang, Y. Cheng, Y. Yang, and B. Póczos. Mmd gan: Towards deeper understanding of moment matching network. NeurIPS, 30, 2017.
[28]
A. Liu, G. Zhang, and J. Lu. Fuzzy time windowing for gradual concept drift adaptation. In FUZZ-IEEE, pages 1--6. IEEE, 2017.
[29]
F. Liu, W. Xu, J. Lu, G. Zhang, A. Gretton, and D. J. Sutherland. Learning deep kernels for non-parametric two-sample tests. In ICML, 2020.
[30]
J. Lu, A. Liu, F. Dong, F. Gu, J. Gama, and G. Zhang. Learning under concept drift: A review. IEEE TKDE, 31(12):2346--2363, 2018.
[31]
K. Miyaguchi and H. Kajino. Cogra: concept-drift-aware stochastic gradient descent for time-series forecasting. In AAAI, 2019.
[32]
A. v. d. Oord, Y. Li, and O. Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
[33]
C. Raab, M. Heusinger, and F.-M. Schleif. Reactive soft prototype computing for concept drift streams. Neurocomputing, 416:340--351, 2020.
[34]
S. Rabanser, S. Günnemann, and Z. Lipton. Failing loudly: An empirical study of methods for detecting dataset shift. NeurIPS, 32, 2019.
[35]
O. Roesler. EEG Eye State. UCI Machine Learning Repository, 2013.
[36]
J. Shao, Z. Ahmadi, and S. Kramer. Prototype-based learning on concept-drifting data streams. In KDD, pages 412--421, 2014.
[37]
Y. Shin, S. Yoon, S. Kim, H. Song, J.-G. Lee, and B. S. Lee. Coherence-based label propagation over time series for accelerated active learning. In ICLR, 2021.
[38]
A. J. Smola, A. Gretton, and K. Borgwardt. Maximum mean discrepancy. In ICONIP, pages 3--6, 2006.
[39]
X. Song, M. Wu, C. Jermaine, and S. Ranka. Statistical change detection for multi-dimensional data. In KDD, pages 667--676, 2007.
[40]
V. M. A. Souza, D. M. Reis, A. G. Maletzke, and G. E. A. P. A. Batista. Challenges in benchmarking stream learning algorithms with real-world data. Data Mining and Knowledge Discovery, 34:1805--1858, 2020.
[41]
P. Trirat, S. Yoon, and J.-G. Lee. MG-TAR: multi-view graph convolutional networks for traffic accident risk prediction. IEEE Transactions on Intelligent Transportation Systems, 24(4):3779--3794, 2023.
[42]
L. Van der Maaten and G. Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(11), 2008.
[43]
A. Van Looveren, J. Klaise, G. Vacanti, O. Cobb, A. Scillitoe, R. Samoilescu, and A. Athorne. Alibi detect: Algorithms for outlier, adversarial and drift detection, 2019.
[44]
Z. Wang, Y. Chen, C. Zhao, Y. Lin, X. Zhao, H. Tao, Y. Wang, and L. Khan. Clear: Contrastive-prototype learning with drift estimation for resource constrained stream mining. In TheWebConf, pages 1351--1362, 2021.
[45]
Z. Wang, Z. Gao, L. Wang, Z. Li, and G. Wu. Boundary-aware cascade networks for temporal action segmentation. In ECCV, pages 34--51. Springer, 2020.
[46]
Z. Wang, L. Liu, Y. Kong, J. Guo, and D. Tao. Online continual learning with contrastive vision transformer. In ECCV, pages 631--650. Springer, 2022.
[47]
A. G. Wilson, Z. Hu, R. Salakhutdinov, and E. P. Xing. Deep kernel learning. In AISTATS, pages 370--378. PMLR, 2016.
[48]
S. Xu and J. Wang. Dynamic extreme learning machine for data stream classification. Neurocomputing, 238:433--449, 2017.
[49]
S. Yoon, H. P. Chan, and J. Han. PDSum: Prototype-driven continuous summarization of evolving multi-document sets stream. In Proceedings of the ACM Web Conference 2023, pages 1650--1661, 2023.
[50]
S. Yoon, D. Lee, Y. Zhang, and J. Han. Unsupervised story discovery from continuous news streams via scalable thematic embedding. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 802--811, 2023.
[51]
S. Yoon, J.-G. Lee, and B. S. Lee. NETS: extremely fast outlier detection from a data stream via set-based processing. Proceedings of the VLDB Endowment, 12(11):1303--1315, 2019.
[52]
S. Yoon, Y. Lee, J.-G. Lee, and B. S. Lee. Adaptive model pooling for online deep anomaly detection from a complex evolving data stream. In KDD. ACM, Aug. 2022.
[53]
S. Yoon, Y. Meng, D. Lee, and J. Han. SCStory: Self-supervised and continual online story discovery. In TheWebConf, pages 1853--1864, 2023.
[54]
H. Yu, J. Li, J. Lu, Y. Song, S. Xie, and G. Zhang. Type-LDD: A type-driven lite concept drift detector for data streams. IEEE TKDE, 2023.
[55]
H. Yu, T. Liu, J. Lu, and G. Zhang. Automatic learning to detect concept drift. arXiv preprint arXiv:2105.01419, 2021.
Index Terms
- Online Drift Detection with Maximum Concept Discrepancy
Index terms have been assigned to the content through auto-classification.
Recommendations
Detecting concept drift: An information entropy based method using an adaptive sliding window
Concept drift in data stream poses many challenges and difficulties in mining this tradition-distinct database. In this paper, we focus on detecting concept drift in evolving data stream. We propose a novel method to detect concept drift using entropy ...
Model-centric transfer learning framework for concept drift detection
AbstractConcept drift refers to the inevitable phenomenon that influences the statistical features of the data stream. Detecting concept drift in data streams quickly and precisely remains challenging, and failure to detect it will render model trained ...
Entropy-based concept drift detection in information systems
AbstractAs time passes, the data within information systems may continuously evolve, causing the target concept to drift. To ensure the effectiveness of data-driven decision making, it is crucial to detect drift in a timely manner and gather relevant ...
Comments
Information & Contributors
Information
Published In
August 2024
6901 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 August 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Funding Sources
- Institute of Information & Communications Technology Planning Evaluation(IITP)
- Korea University
Conference
KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 25 - 29, 2024
Barcelona, Spain
Acceptance Rates
Overall Acceptance Rate 1,133 of 8,635 submissions, 13%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 357Total Downloads
- Downloads (Last 12 months)357
- Downloads (Last 6 weeks)97
Reflects downloads up to 04 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in