skip to main content
10.1145/3468264.3468548acmconferencesArticle/Chapter ViewAbstractPublication PagesfseConference Proceedingsconference-collections
research-article
Open access

Timely and accurate detection of model deviation in self-adaptive software-intensive systems

Published: 18 August 2021 Publication History

Abstract

Control-based approaches to self-adaptive software-intensive systems (SASs) are hailed for their optimal performance and theoretical guarantees on the reliability of adaptation behavior. However, in practice the guarantees are often threatened by model deviations occurred at runtime. In this paper, we propose a Model-guided Deviation Detector (MoD2) for timely and accurate detection of model deviations. To ensure reliability, a SAS can switch a control-based optimal controller for a mandatory controller once an unsafe model deviation is detected. MoD2 achieves both high timeliness and high accuracy through a deliberate fusion of parameter deviation estimation, uncertainty compensation, and safe region quantification. Empirical evaluation with three exemplar systems validated the efficacy of MoD2 (93.3% shorter detection delay, 39.4% lower FN rate, and 25.2% lower FP rate), as well as the benefits of the adaptation-switching mechanism (abnormal rate dropped by 29.2%).

References

[1]
Sridhar Adepu and Aditya Mathur. 2016. An investigation into the response of a water treatment system to cyber attacks. In Proceedings of 17th IEEE International Symposium on High Assurance Systems Engineering. 141–148. https://doi.org/10.1109/HASE.2016.14
[2]
Cristina Alcaraz and Stephen Wolthusen. 2014. Recovery of structural controllability for control systems. In International Conference on Critical Infrastructure Protection. 47–63. https://doi.org/10.1007/978-3-662-45355-1
[3]
Nadia Alshahwan and Mark Harman. 2011. Automated web application testing using search based software engineering. In Proceedings of the 26th IEEE/ACM International Conference on Automated Software Engineering. 3–12. https://doi.org/10.1109/ASE.2011.6100082
[4]
Konstantinos Angelopoulos, Alessandro Papadopoulos, Vítor Silva Souza, and John Mylopoulos. 2016. Model predictive control for software systems with CobRA. In Proceedings of the 11th International Symposium on Software Engineering for Adaptive and Self-Managing Systems. 35–46. https://doi.org/10.1145/2897053.2897054
[5]
Konstantinos Angelopoulos, Alessandro Papadopoulos, Vítor Silva Souza, and John Mylopoulos. 2018. Engineering self-adaptive software systems: From requirements to model predictive control. ACM Transactions on Autonomous and Adaptive Systems, 13, 1 (2018), 1–27. https://doi.org/10.1145/3105748
[6]
Martin Arlitt and Tai Jin. 2000. A workload characterization study of the 1998 world cup web site. IEEE Network, 14, 3 (2000), 30–37. https://doi.org/10.1109/65.844498
[7]
Martin Arlitt and Carey Williamson. 1996. Web server workload characterization: The search for invariants. ACM SIGMETRICS Performance Evaluation Review, 24, 1 (1996), 126–137. https://doi.org/10.1145/233008.233034
[8]
Kaung Myat Aung. 2015. Secure water treatment testbed (SWaT): an overview. Singapore University of Technology and Design.
[9]
Abhijit Badwe, Ravindra Gudi, Rohit Patwardhan, Sirish Shah, and Sachin Patwardhan. 2009. Detection of model-plant mismatch in MPC applications. Journal of Process Control, 19, 8 (2009), 1305–1313. https://doi.org/10.1016/j.jprocont.2009.04.007
[10]
Saeid Barati, Ferenc Bartha, Swarnendu Biswas, Robert Cartwright, Adam Duracz, Donald Fussell, and et al. 2019. Proteus: Language and runtime support for self-adaptive software development. IEEE Software, 36, 2 (2019), 73–82. https://doi.org/10.1109/MS.2018.2884864
[11]
Luciano Baresi, Sam Guinea, Alberto Leva, and Giovanni Quattrocchi. 2016. A discrete-time feedback controller for containerized cloud applications. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 217–228. https://doi.org/10.1145/2950290.2950328
[12]
Michele Basseville and Igor Nikiforov. 1993. Detection of abrupt changes: theory and application. 104, https://doi.org/10.5555/151741
[13]
Mogens Blanke, Michel Kinnaert, Jan Lunze, Marcel Staroswiecki, and Jochen Schröder. 2006. Diagnosis and fault-tolerant control. 2, Springer. https://doi.org/10.1007/978-3-540-35653-0
[14]
Yuriy Brun, Giovanna Di Marzo Serugendo, Cristina Gacek, Holger Giese, Holger Kienle, Marin Litoiu, and et al. 2009. Engineering self-adaptive systems through feedback loops. In Software Engineering for Self-Adaptive Systems. Springer, 48–70. https://doi.org/10.1007/978-3-642-02161-9_3
[15]
Laurentiu Checiu, Bogdan Solomon, Dan Ionescu, Marin Litoiu, and Gabriel Iszlai. 2011. Observability and controllability of autonomic computing systems for composed web services. In Proceedings of the 6th IEEE International Symposium on Applied Computational Intelligence and Informatics. 269–274. https://doi.org/10.1109/SACI.2011.5873012
[16]
Xing Chen, Shihong Chen, Yun Ma, Bichun Liu, Ying Zhang, and Gang Huang. 2019. An adaptive offloading framework for Android applications in mobile edge computing. Science China Information Sciences, 62, 8 (2019), 1–17. https://doi.org/10.1007/s11432-018-9749-8
[17]
Yuqi Chen, Poskitt Christopher, and Jun Sun. 2018. Learning from mutants: Using code mutation to learn and monitor invariants of a cyber-physical system. In Proceedings of the 39th IEEE Symposium on Security and Privacy. 648–660. https://doi.org/10.1109/SP.2018.00016
[18]
OW2 Consortium. 2008. Rubis: Rice university bidding system. URL http://rubis. ow2. org.
[19]
Mário de Castro and Ignacio Vidal. 2019. Bayesian inference in measurement error models from objective priors for the bivariate normal distribution. Statistical Papers, 60, 4 (2019), 1059–1078. https://doi.org/10.1007/s00362-016-0863-7
[20]
Naeem Esfahani, Ehsan Kouroshfar, and Sam Malek. 2011. Taming uncertainty in self-adaptive software. In Proceedings of the 19th ACM SIGSOFT Symposium on the Foundations of Software Engineering and 13th European Software Engineering Conference. 234–244. https://doi.org/10.1145/2025113.2025147
[21]
Oreste Fecarotta, Riccardo Martino, and Cristina Morani. 2019. Wastewater pump control under mechanical wear. Water, 11, 6 (2019), 1210. https://doi.org/10.3390/w11061210
[22]
Antonio Filieri, Carlo Ghezzi, Alberto Leva, and Martina Maggio. 2011. Self-adaptive software meets control theory: A preliminary approach supporting reliability requirements. In Proceedings of 26th IEEE/ACM International Conference on Automated Software Engineering. 283–292. https://doi.org/10.1109/ASE.2011.6100064
[23]
Antonio Filieri, Henry Hoffmann, and Martina Maggio. 2014. Automated design of self-adaptive software with control-theoretical formal guarantees. In Proceedings of the 36th International Conference on Software Engineering. 299–310. https://doi.org/10.1145/2568225.2568272
[24]
Antonio Filieri, Henry Hoffmann, and Martina Maggio. 2015. Automated multi-objective control for self-adaptive software design. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering. 13–24. https://doi.org/10.1145/2786805.2786833
[25]
Antonio Filieri, Martina Maggio, Konstantinos Angelopoulos, Nicolás D’Ippolito, Ilias Gerostathopoulos, Andreas Berndt Hempel, and et al. 2015. Software engineering meets control theory. In Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems. 71–82. https://doi.org/10.1109/SEAMS.2015.12
[26]
Antonio Filieri, Martina Maggio, Konstantinos Angelopoulos, Nicolás D’ippolito, Ilias Gerostathopoulos, Andreas Hempel, and et al. 2017. Control strategies for self-adaptive software systems. ACM Transactions on Autonomous and Adaptive Systems, 11, 4 (2017), 1–31. https://doi.org/10.1145/3024188
[27]
David Garlan, S-W Cheng, A-C Huang, Bradley Schmerl, and Peter Steenkiste. 2004. Rainbow: Architecture-based self-adaptation with reusable infrastructure. Computer, 37, 10 (2004), 46–54. https://doi.org/10.1109/MC.2004.175
[28]
Carlo Ghezzi, Leandro Sales Pinto, Paola Spoletini, and Giordano Tamburrelli. 2013. Managing non-functional uncertainty via model-driven adaptivity. In Proceedings of the 35th International Conference on Software Engineering. 33–42. https://doi.org/10.1109/ICSE.2013.6606549
[29]
Jayanta K Ghosh, Mohan Delampady, and Tapas Samanta. 2007. An introduction to Bayesian analysis: theory and methods. Springer Science & Business Media. https://doi.org/10.1007/978-0-387-35433-0
[30]
Graham Goodwin, Stefan Graebe, and Mario Salgado. 2001. Control system design. https://doi.org/10.5555/557291
[31]
Jorn Gruber, Daniel Ramirez, Teodoro Alamo, and Eduardo Camacho. 2011. Min–Max MPC based on an upper bound of the worst case cost with guaranteed stability. Journal of Process Control, 21, 1 (2011), 194–204. https://doi.org/10.1016/j.jprocont.2010.11.016
[32]
Fredrik Gustafsson and Fredrik Gustafsson. 2000. Adaptive filtering and change detection. 1, Citeseer. https://doi.org/10.1002/0470841613
[33]
Farshad Harirchi, Zheng Luo, and Necmiye Ozay. 2016. Model (in)validation and fault detection for systems with polynomial state-space models. In American Control Conference. 1017–1023. https://doi.org/10.1109/ACC.2016.7525048
[34]
Farshad Harirchi and Necmiye Ozay. 2018. Guaranteed model-based fault detection in cyber physical systems: A model invalidation approach. Automatica, 93 (2018), 476–488. https://doi.org/10.1016/j.automatica.2018.03.040
[35]
Zhijian He, Yao, Enyan Huang, Qixin Wang, Yu Pei, and Haidong Yuan. 2019. A system identification based oracle for control-cps software fault localization. In Proceedings of the 41st International Conference on Software Engineering. 116–127. https://doi.org/10.1109/ICSE.2019.00029
[36]
Joseph Hellerstein, Yixin Diao, Sujay Parekh, and Dawn Tilbury. 2004. Feedback control of computing systems. Wiley Online Library. https://doi.org/10.1002/047166880X
[37]
Emilio Incerto, Mirco Tribastone, and Catia Trubiani. 2017. Software performance self-adaptation through efficient model predictive control. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering. 485–496. https://doi.org/10.1109/ASE.2017.8115660
[38]
Rolf Isermann. 2013. Digital control systems. Springer Science & Business Media. https://doi.org/10.1007/978-3-642-86420-9
[39]
Hengle Jiang, Sebastian Elbaum, and Carrick Detweiler. 2013. Reducing failure rates of robotic systems though inferred invariants monitoring. In Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems. 1899–1906. https://doi.org/10.1109/IROS.2013.6696608
[40]
Michael Johnson and Mohammad Moradi. 2005. PID control. Springer. https://doi.org/10.1007/1-84628-148-2
[41]
Raphael Jungers, Atreyee Kundu, and Maurice Heemels. 2017. Observability and controllability analysis of linear systems subject to data losses. IEEE Trans. Automat. Control, 63, 10 (2017), 3361–3376. https://doi.org/10.1109/TAC.2017.2781374
[42]
Eunsuk Kang, Sridhar Adepu, Daniel Jackson, and Aditya Mathur. 2016. Model-based security analysis of a water treatment system. In Proceedings of the 2nd International Workshop on Software Engineering for Smart Cyber-Physical Systems. 22–28. https://doi.org/10.1145/2897035.2897041
[43]
Youngjoo Kim and Hyochoong Bang. 2018. Introduction to Kalman filter and its applications. Introduction and Implementations of the Kalman Filter, 1 (2018), 1–16. https://doi.org/10.5772/intechopen.80600
[44]
Cristian Klein, Martina Maggio, Karl-Erik Årzén, and Francisco Hernández-Rodriguez. 2014. Brownout: Building more robust cloud applications. In Proceedings of the 36th International Conference on Software Engineering. 700–711. https://doi.org/10.1145/2568225.2568227
[45]
Qiang Li, Ranyang Li, Kaifan Ji, and Wei Dai. 2015. Kalman filter and its application. In Proceedings of the 8th International Conference on Intelligent Networks and Intelligent Systems. 74–77. https://doi.org/10.1109/ICINIS.2015.35
[46]
Dan Ling, Ying Zheng, Hong Zhang, Weidong Yang, and Bo Tao. 2017. Detection of model-plant mismatch in closed-loop control system. Journal of Process Control, 57 (2017), 66–79. https://doi.org/10.1016/j.jprocont.2017.06.008
[47]
Marin Litoiu, Mary Shaw, Gabriel Tamura, Norha Villegas, Hausi Müller, Holger Giese, and et al. 2017. What can control theory teach us about assurances in self-adaptive software systems? In Software Engineering for Self-Adaptive Systems III. Assurances. Springer, 90–134. https://doi.org/10.1007/978-3-319-74183-3
[48]
Lennart Ljung. 1999. System identification. Wiley encyclopedia of electrical and electronics engineering, 1–19. https://doi.org/10.1007/978-1-4612-1768-8_11
[49]
Martina Maggio, Alessandro Vittorio Papadopoulos, Antonio Filieri, and Henry Hoffmann. 2017. Automated control of multiple software goals using multiple actuators. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering. 373–384. https://doi.org/10.1145/3106237.3106247
[50]
Martina Maggio, Alessandro Vittorio Papadopoulos, Antonio Filieri, and Henry Hoffmann. 2017. Self-adaptive video encoder: Comparison of multiple adaptation strategies made simple. In Proceedings of 12th IEEE/ACM International Symposium on Software Engineering for Adaptive and Self-Managing Systems. 123–128. https://doi.org/10.1109/SEAMS.2017.16
[51]
Claudio Mandrioli and Martina Maggio. 2020. Testing self-adaptive software with probabilistic guarantees on performance metrics. In Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 1002–1014. https://doi.org/10.1145/3368089.3409685
[52]
Claudio Menghi, Shiva Nejati, Lionel Briand, and Yago Isasi Parache. 2020. Approximation-refinement testing of compute-intensive cyber-physical models: An approach based on system identification. In Proceedings of the 42nd International Conference on Software Engineering. 372–384. https://doi.org/10.1145/3377811.3380370
[53]
Douglas Montgomery. 2020. Introduction to statistical quality control. John Wiley & Sons. https://doi.org/10.1002/9781119528425
[54]
Gabriel A Moreno, Javier Cámara, David Garlan, and Bradley Schmerl. 2015. Proactive self-adaptation under uncertainty: a probabilistic model checking approach. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering. 1–12. https://doi.org/10.1145/2786805.2786853
[55]
Alexander Palm, Andreas Metzger, and Klaus Pohl. 2020. Online reinforcement learning for self-adaptive information systems. In Proceedings of the 32nd International Conference on Advanced Information Systems Engineering. 169–184. https://doi.org/10.1007/978-3-030-49435-3_11
[56]
Tharindu Patikirikorala, Alan Colman, Jun Han, and Liuping Wang. 2012. A systematic survey on the design of self-adaptive software systems using control engineering approaches. In Proceedings of the 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems. 33–42. https://doi.org/10.1109/SEAMS.2012.6224389
[57]
Yi Qin, Tao Xie, Chang Xu, Angello Astorga, and Jian Lu. 2019. CoMID: Context-Based Multiinvariant Detection for Monitoring Cyber-Physical Software. IEEE Transactions on Reliability, 69, 1 (2019), 106–123. https://doi.org/10.1109/TR.2019.2933324
[58]
Mazeiar Salehie and Ladan Tahvildari. 2012. Towards a goal-driven approach to action selection in self-adaptive software. Software: Practice and Experience, 42, 2 (2012), 211–233. https://doi.org/10.1002/spe.1066
[59]
Stepan Shevtsov, Mihaly Berekmeri, Danny Weyns, and Martina Maggio. 2017. Control-theoretical software adaptation: A systematic literature review. IEEE Transactions on Software Engineering, 44, 8 (2017), 784–810. https://doi.org/10.1109/TSE.2017.2704579
[60]
Stepan Shevtsov and Danny Weyns. 2016. Keep it simplex: Satisfying multiple goals with guarantees in control-based self-adaptive systems. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. 229–241. https://doi.org/10.1145/2950290.2950301
[61]
Stepan Shevtsov, Danny Weyns, and Martina Maggio. 2019. SimCA*: A Control-theoretic approach to handle uncertainty in self-adaptive systems with guarantees. ACM Transactions on Autonomous and Adaptive Systems, 13, 4 (2019), 1–34. https://doi.org/10.1145/3328730
[62]
Silvio Simani, Cesare Fantuzzi, and Ronald Jon Patton. 2003. Model-based fault diagnosis techniques. In Model-based Fault Diagnosis in Dynamic Systems Using Identification Techniques. Springer, 19–60. https://doi.org/10.1007/978-1-4471-3829-7_2
[63]
Hui Song, Amit Raj, Saeed Hajebi, Aidan Clarke, and Siobhán Clarke. 2013. Model-based cross-layer monitoring and adaptation of multilayer systems. Science China Information Sciences, 56, 8 (2013), 1–15. https://doi.org/10.1007/s11432-013-4915-5
[64]
Guojun Wang and Siva Sivaganesan. 2013. Objective priors for parameters in a normal linear regression with measurement error. Communications in Statistics-Theory and Methods, 42, 15 (2013), 2694–2713. https://doi.org/10.1080/03610926.2011.608476
[65]
Xiaoyun Zhu, Mustafa Uysal, Zhikui Wang, Sharad Singhal, Arif Merchant, Pradeep Padala, and et al. 2009. What does control theory bring to systems research? ACM SIGOPS Operating Systems Review, 43, 1 (2009), 62–69. https://doi.org/10.1145/1496909.1496922

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
August 2021
1690 pages
ISBN:9781450385626
DOI:10.1145/3468264
This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 August 2021

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. Control Theory
  2. Model Deviation
  3. Self-Adaptive Software

Qualifiers

  • Research-article

Funding Sources

Conference

ESEC/FSE '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)213
  • Downloads (Last 6 weeks)31
Reflects downloads up to 17 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media