skip to main content
research-article

Semi-Supervised Range-Based Anomaly Detection for Cloud Systems

Published: 01 June 2023 Publication History

Abstract

The inherent characteristics of cloud systems often lead to anomalies, which pose challenges for high availability, reliability, and high performance. Detecting anomalies in cloud key performance indicators (KPI) is a critical step towards building a secure and trustworthy system with early mitigation features. This work is motivated by (i) the efficacy of recent reconstruction-based anomaly detection (AD), (ii) the misrepresentation of the accuracy of time series anomaly detection because point-based <italic>Precision</italic> and <italic>Recall</italic> are used to evaluate the efficacy for range-based anomalies, and (iii) detects performance and security anomalies when distributions shift and overlaps. In this paper, we propose a novel semi-supervised dynamic density-based detection rule that uses the reconstruction error vectors in order to detect anomalies. We use long short-term memory networks based on encoder-decoder (LSTM-ED) architecture to reconstruct the normal KPI time series. We experiment with both testbed and a diverse set of real-world datasets. The experimental results show that the dynamic density approach exhibits better performance compared to other detection rules using both standard and range-based evaluation metrics. We also compare the performance of our approach with state-of-the-art methods, outperforms in detecting both performance and security anomalies.

References

[1]
Cloud 2025: The future of workloads in a cloud-first, post-Covid-19 world.” LogicMonitor. 2021. Accessed: Mar. 29, 2022. [Online]. Available: https://www.logicmonitor.com/resource/cloud-2025
[2]
J. Hochenbaum, O. S. Vallis, and A. Kejariwal, “Automatic anomaly detection in the cloud via statistical learning,” 2017, arXiv:1704.07706.
[3]
O. Ibidunmoye, A. Rezaie, and E. Elmroth, “Adaptive anomaly detection in performance metric streams,” IEEE Trans. Netw. Service Manag., vol. 15, no. 1, pp. 217–231, Mar. 2018.
[4]
X. Zhanget al., “Cross-dataset time series anomaly detection for cloud systems,” in Proc. USENIX Conf. Usenix Annu. Tech. Conf., 2019, pp. 1063–1076.
[5]
P. K. Deka, M. H. Bhuyan, Y. Kadobayashi, and E. Elmroth, “Adversarial impact on anomaly detection in cloud datacenters,” in Proc. IEEE 24th Pacific Rim Int. Symp. Dependable Comput. (PRDC), 2019, pp. 188–197.
[6]
Global enterprise server hourly downtime cost 2019.” Statista. 2020. Accessed: Mar. 29, 2022. [Online]. Available: https://www.statista.com/statistics/753938/world-wide-enterprise-server-hourly-downtime-cost/
[7]
T. Wang, W. Zhang, C. Ye, J. Wei, H. Zhong, and T. Huang, “FD4C: Automatic fault diagnosis framework for Web applications in cloud computing,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 46, no. 1, pp. 61–75, Jan. 2016.
[8]
V. Chandola, A. Banerjee, and V. Kumar, “Anomaly detection: A survey,” ACM Comput. Surveys, vol. 41, no. 3, pp. 1–58, 2009.
[9]
N. Tatbul, T. J. Lee, S. Zdonik, M. Alam, and J. Gottschlich, “Precision and recall for time series,” in Proc. Adv. Neural Inf. Process. Syst., 2018, pp. 1920–1930.
[10]
S. Ahmad, A. Lavin, S. Purdy, and Z. Agha, “Unsupervised real-time anomaly detection for streaming data,” Neurocomputing, vol. 262, pp. 134–147, Nov. 2017.
[11]
M. H. Bhuyan, D. K. Bhattacharyya, and J. K. Kalita, “Network anomaly detection: Methods, systems and tools,” IEEE Commun. Surveys Tuts., vol. 16, no. 1, pp. 303–336, 1st Quart., 2014.
[12]
X. Xu, H. Liu, and M. Yao, “Recent progress of anomaly detection,” Complexity, vol. 2019, Jan. 2019, Art. no. [Online]. Available: https://doi.org/10.1155/2019/2686378
[13]
I. Sutskever, O. Vinyals, and Q. V. Le, “Sequence to sequence learning with neural networks,” in Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 3104–3112.
[14]
K. Choet al., “Learning phrase representations using RNN encoder-decoder for statistical machine translation,” 2014, arXiv:1406.1078.
[15]
O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 3156–3164.
[16]
R. Prabhavalkar, K. Rao, T. N. Sainath, B. Li, L. Johnson, and N. Jaitly, “A comparison of sequence-to-sequence models for speech recognition,” in Proc. Interspeech, 2017, pp. 939–943.
[17]
P. Liu, Z. Zeng, and J. Wang, “Multiple Mittag–Leffler stability of fractional-order recurrent neural networks,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 47, no. 8, pp. 2279–2288, Aug. 2017.
[18]
S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Comput., vol. 9, no. 8, pp. 1735–1780, 1997.
[19]
I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. Cambridge, MA, USA: MIT Press, 2016.
[20]
P. Malhotra, A. Ramakrishnan, G. Anand, L. Vig, P. Agarwal, and G. Shroff, “LSTM-based encoder-decoder for multi-sensor anomaly detection,” 2016, arXiv:1607.00148.
[21]
C. Yin, S. Zhang, J. Wang, and N. N. Xiong, “Anomaly detection based on convolutional recurrent autoencoder for IoT time series,” IEEE Trans. Syst., Man, Cybern., Syst., vol. 52, no. 1, pp. 112–122, Jan. 2022.
[22]
J. Goh, S. Adepu, M. Tan, and Z. S. Lee, “Anomaly detection in cyber physical systems using recurrent neural networks,” in Proc. IEEE 18th Int. Symp. High Assurance Syst. Eng. (HASE), 2017, pp. 140–145.
[23]
T.-Y. Kim and S.-B. Cho, “Web traffic anomaly detection using C-LSTM neural networks,” Expert Syst. Appl., vol. 106, pp. 66–76, Sep. 2018.
[24]
K. Hundman, V. Constantinou, C. Laporte, I. Colwell, and T. Soderstrom, “Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding,” in Proc. 24th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., 2018, pp. 387–395.
[25]
N. Davis, G. Raina, and K. Jagannathan, “A framework for end-to-end deep learning-based anomaly detection in transportation networks,” Transp. Res. Interdiscip. Perspectives, vol. 5, pp. 100–112, May 2020.
[26]
P. Malhotraet al., “Multi-sensor prognostics using an unsupervised health index based on LSTM encoder-decoder,” 2016, arXiv:1608.06154.
[27]
R. Chalapathy and S. Chawla, “Deep learning for anomaly detection: A survey,” 2019, arXiv:1901.03407.
[28]
L. Bontemps, J. McDermott, N.-A. Le-Khac, and V. L. Cao, “Collective anomaly detection based on long short-term memory recurrent neural networks,” in Proc. Int. Conf. Future Data Security Eng., 2016, pp. 141–152.
[29]
C. Wang, K. Viswanathan, L. Choudur, V. Talwar, W. Satterfield, and K. Schwan, “Statistical techniques for online anomaly detection in data centers,” in Proc. 12th IFIP/IEEE Int. Symp. Int. Netw. Manage. (IM) Workshops, 2011, pp. 385–392.
[30]
N. Gruschka and M. Jensen, “Attack surfaces: A taxonomy for attacks on cloud services,” in Proc. IEEE 3rd Int. Conf. Cloud Comput., 2010, pp. 276–279.
[31]
A. K. Marnerides, M. R. Watson, N. Shirazi, A. Mauthe, and D. Hutchison, “Malware analysis in cloud computing: Network and system characteristics,” in Proc. IEEE Globecom Workshops (GC Wkshps), 2013, pp. 482–487.
[32]
A. K. Marnerides, P. Spachos, P. Chatzimisios, and A. U. Mauthe, “Malware detection in the cloud under ensemble empirical mode decomposition,” in Proc. Int. Conf. Comput., Netw. Commun. (ICNC), 2015, pp. 82–88.
[33]
S. Roschke, F. Cheng, and C. Meinel, “Intrusion detection in the cloud,” in Proc. 8th IEEE Int. Conf. Dependable, Auton. Secure Comput., 2009, pp. 729–734.
[34]
H. S. Pannu, J. Liu, and S. Fu, “AAD: Adaptive anomaly detection system for cloud computing infrastructures,” in Proc. IEEE 31st Symp. Reliable Distrib. Syst., 2012, pp. 396–397.
[35]
Q. Guan, S. Fu, N. DeBardeleben, and S. Blanchard, “Exploring time and frequency domains for accurate and automated anomaly detection in cloud computing systems,” in Proc. IEEE 19th Pacific Rim Int. Symp. Dependable Comput., 2013, pp. 196–205.
[36]
M. R. Watson, N.-U.-H. Shirazi, A. K. Marnerides, A. Mauthe, and D. Hutchison, “Malware detection in cloud computing infrastructures,” IEEE Trans. Dependable Secure Comput., vol. 13, no. 2, pp. 192–205, Mar./Apr. 2016.
[37]
M. Abdelsalam, R. Krishnan, Y. Huang, and R. Sandhu, “Malware detection in cloud infrastructures using convolutional neural networks,” in Proc. IEEE 11th Int. Conf. Cloud Comput. (CLOUD), 2018, pp. 162–169.
[38]
Q. Zhang, R. Han, G. Xin, C. H. Liu, G. Wang, and L. Y. Chen, “Lightweight and accurate DNN-based anomaly detection at edge,” IEEE Trans. Parallel Distrib. Syst., vol. 33, no. 11, pp. 2927–2942, Nov. 2022.
[39]
O. Ibidunmoye, F. Hernández-Rodriguez, and E. Elmroth, “Performance anomaly detection and bottleneck identification,” ACM Comput. Surveys, vol. 48, no. 1, pp. 1–35, 2015.
[40]
W.-S. Hwang, J.-H. Yun, J. Kim, and H. C. Kim, “Time-series aware precision and recall for anomaly detection: Considering variety of detection result and addressing ambiguous Labeling,” in Proc. CIKM, 2019, pp. 2241–2244.
[41]
X. Zhou and A. Del Valle, “Range based confusion matrix for imbalanced time series classification,” in Proc. 6th Conf. Data Sci. Mach. Learn. Appl. (CDMA), 2020, pp. 1–6.
[42]
C. Sauvanaud, M. Kaâniche, K. Kanoun, K. Lazri, and G. D. S. Silvestre, “Anomaly detection and diagnosis for cloud services: Practical experiments and lessons learned,” J. Syst. Softw., vol. 139, pp. 84–106, May 2018.
[43]
C. Monni, M. Pezzè, and G. Prisco, “An RBM anomaly detector for the cloud,” in Proc. 12th IEEE Conf. Softw. Testing, Validation Verification (ICST), 2019, pp. 148–159.
[44]
Q. Heet al., “A game-theoretical approach for MitigatingEdge DDoS attack,” IEEE Trans. Dependable Secure Comput., vol. 19, no. 4, pp. 2333–2348, Jul./Aug. 2022. [Online]. Available: https://doi.org/10.1109/TDSC.2021.3055559
[45]
M. A. Salahuddin, V. Pourahmadi, H. A. Alameddine, M. F. Bari, and R. Boutaba, “Chronos: DDoS attack detection using time-based autoencoder,” IEEE Trans. Netw. Service Manag., vol. 19, no. 1, pp. 627–641, Mar. 2022.
[46]
A. Siffer, P.-A. Fouque, A. Termier, and C. Largouet, “Anomaly detection in streams with extreme value theory,” in Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Disc. Data Mining, 2017, pp. 1067–1075.
[47]
E. Grafarend, Linear and Nonlinear Models: Fixed Effects, Random Effects, and Mixed Models. Berlin, Germany: Walter de Gruyter, 2006.
[48]
M. Solaimani, M. Iftekhar, L. Khan, and B. Thuraisingham, “Statistical technique for online anomaly detection using spark over heterogeneous data from multi-source vmware performance data,” in Proc. IEEE Int. Conf. Big Data (Big Data), 2014, pp. 1086–1094.
[49]
G. Smrithy and R. Balakrishnan, “A statistical technique for online anomaly detection for big data streams in cloud collaborative environment,” in Proc. IEEE Int. Conf. Comput. Inf. Technol. (CIT), 2016, pp. 108–111.
[50]
M. Thill, W. Konen, and T. Bäck, “Online anomaly detection on the webscope S5 dataset: A comparative study,” in Proc. Evolving Adapt. Intell. Syst., May 2017, pp. 1–8.
[51]
M. M. Breunig, H.-P. Kriegel, R. T. Ng, and J. Sander, “LOF: Identifying density-based local outliers,” SIGMOD Rec., vol. 29, no. 2, pp. 93–104, May 2000.
[52]
N. Chinchor, “MUC-4 evaluation metrics,” in Proc. 4th Conf. Message Understanding, 1992, pp. 22–29.
[53]
H. Sak, A. Senior, and F. Beaufays, “Long short-term memory based recurrent neural network architectures for large vocabulary speech recognition,” 2014, arXiv:1402.1128.
[54]
N. Laptev, S. Amizadeh, and I. Flint. “Online dataset for anomaly detection.” 2015. Accessed: Mar. 25, 2022. [Online]. Available: https://webscope.sandbox.yahoo.com/catalog.php?datatype=s&did=70
[55]
N. Laptev, S. Amizadeh, and I. Flint, “Generic and scalable framework for automated time-series anomaly detection,” in Proc. 21th ACM SIGKDD Int. Conf. Knowl. Disc. Data Min., 2015, pp. 1939–1947.
[56]
I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, “Toward generating a new intrusion detection Dataset and intrusion traffic Characterization,” in Proc. ICISSP, 2018, pp. 1–9.
[57]
E. Cecchet, J. Marguerite, and W. Zwaenepoel, “Performance and scalability of EJB applications,” SIGPLAN Not., vol. 37, no. 11, pp. 246–261, Nov. 2002.
[58]
S. Avallone, S. Guadagno, D. Emma, A. Pescape, and G. Ventre, “D-ITG distributed Internet traffic generator,” in Proc. 1st Int. Conf. Quantitative Eval. Syst. (QEST), 2002, pp. 316–317.
[59]
I. Sharafaldin, A. H. Lashkari, S. Hakak, and A. A. Ghorbani, “Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy,” in Proc. Int. Carnahan Conf. Security Technol. (ICCST), 2019, pp. 1–8.
[60]
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” 2014, arXiv:1412.6980.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Network and Service Management
IEEE Transactions on Network and Service Management  Volume 20, Issue 2
June 2023
1224 pages

Publisher

IEEE Press

Publication History

Published: 01 June 2023

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 22 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media