research-article

On minimising the maximum expected verification time

Authors:

Annalisa Massini,

Enrico TronciAuthors Info & Claims

Information Processing Letters, Volume 122, Issue C

Pages 8 - 16

https://doi.org/10.1016/j.ipl.2017.02.001

Published: 01 June 2017 Publication History

Abstract

Cyber Physical Systems (CPSs) consist of hardware and software components. To verify that the whole (i.e., software + hardware) system meets the given specifications, exhaustive simulation-based approaches (Hardware In the Loop Simulation, HILS) can be effectively used by first generating all relevant simulation scenarios (i.e., sequences of disturbances) and then actually simulating all of them (verification phase). When considering the whole verification activity, we see that the above mentioned verification phase is repeated until no error is found. Accordingly, in order to minimise the time taken by the whole verification activity, in each verification phase we should, ideally, start by simulating scenarios witnessing errors (counterexamples). Of course, to know beforehand the set of such scenarios is not feasible. In this paper we show how to select scenarios so as to minimise the Worst Case Expected Verification Time. Simulation: main workhorse for System Level Verification of Cyber Physical Systems.A counterexample is a simulation scenario witnessing a design error.WCEVT: max expected number of scenarios simulated before hitting a counterexample.We show that the minimum WCEVT is (n+1)/2 (n number of simulation scenarios).The infinite set of optimal simulation strategies forms a convex bounded polytope.

References

[1]

H. Abbas, G. Fainekos, S. Sankaranarayanan, F. Ivani, A. Gupta, Probabilistic temporal logic falsification of cyber-physical systems, ACM Trans. Embed. Comput. Syst., 12 (2013).

Digital Library

[2]

H. Abbas, B. Hoxha, G. Fainekos, K. Ueda, Robustness-guided temporal logic testing and verification for stochastic cyber-physical systems, in: Proc. IEEE CYBER 2014, IEEE, 2014.

[3]

A. Adir, E. Almog, L. Fournier, E. Marcus, M. Rimon, M. Vinov, A. Ziv, Genesys-Pro: innovations in test program generation for functional processor verification, IEEE Des. Test Comput., 21 (2004).

Digital Library

[4]

V. Alimguzhin, F. Mari, I. Melatti, I. Salvo, E. Tronci, Automatic control software synthesis for quantized discrete time hybrid systems, in: Proc. CDC 2012, IEEE, 2012.

[5]

V. Alimguzhin, F. Mari, I. Melatti, I. Salvo, E. Tronci, On model based synthesis of embedded control software, in: Proc. EMSOFT 2012, ACM, 2012.

Digital Library

[6]

V. Alimguzhin, F. Mari, I. Melatti, I. Salvo, E. Tronci, A map-reduce parallel approach to automatic synthesis of control software, in: Lect. Notes Comput. Sci., vol. 7976, Springer, 2013.

[7]

V. Alimguzhin, F. Mari, I. Melatti, I. Salvo, E. Tronci, On-the-fly control software synthesis, in: Lect. Notes Comput. Sci., vol. 7976, Springer, 2013.

[8]

R. Alur, Formal verification of hybrid systems, in: Proc. EMSOFT 2011, ACM, 2011.

Digital Library

[9]

A. Arcuri, M. Iqbal, L. Briand, Random testing: theoretical results and practical implications, IEEE Trans. Softw. Eng., 38 (2012).

Digital Library

[10]

C. Browne, E. Powley, D. Whitehouse, S. Lucas, P. Cowling, P. Rohlfshagen, S. Tavener, D. Liebana, S. Samothrakis, S. Colton, A survey of Monte Carlo tree search methods, IEEE Trans. Comput. Intell. AI Games, 4 (2012).

[11]

E. Clarke, A. Donz, A. Legay, On simulation-based probabilistic model checking of mixed-analog circuits, Form. Methods Syst. Des., 36 (2010).

Digital Library

[12]

E. Clarke, O. Grumberg, D. Peled, Model Checking, MIT, 1999.

Digital Library

[13]

E. Clarke, T. Henzinger, H. Veith, Handbook of Model Checking, Springer, 2016.

[14]

R. Dechter, K. Kask, E. Bin, R. Emek, Generating random solutions for constraint satisfaction problems, in: Proc. AAAI 2002, AAAI, 2002.

Digital Library

[15]

G. Della Penna, B. Intrigila, I. Melatti, E. Tronci, M. Venturini Zilli, Finite horizon analysis of Markov chains with the Murphi verifier, Int. J. Softw. Tools Technol. Transf., 8 (2006).

Digital Library

[16]

A. Dokhanchi, A. Zutshi, R. Sriniva, S. Sankaranarayanan, G. Fainekos, Requirements driven falsification with coverage metrics, in: Proc. EMSOFT 2015, IEEE, 2015.

Digital Library

[17]

P. Duggirala, S. Mitra, M. Viswanathan, M. Potok, C2E2: a verification tool for stateflow models, in: Lect. Notes Comput. Sci., vol. 9035, Springer, 2015.

Digital Library

[18]

C. Grimm, C. Radojicic, Verification and validation of AMS systems: towards coverage of uncertainties, in: Proc. IMSTW 2015, IEEE, 2015.

[19]

R. Grosu, S. Smolka, Monte Carlo model checking, in: Lect. Notes Comput. Sci., vol. 3440, Springer, 2005.

Digital Library

[20]

B. Hayes, I. Melatti, T. Mancini, M. Prodanovic, E. Tronci, Residential demand management using individualised demand aware price policies, IEEE Trans. Smart Grid (2016).

[21]

D. Jansen, J. Katoen, M. Oldenkamp, M. Stoelinga, I. Zapreev, How fast and fat is your probabilistic model checker? An experimental performance comparison, in: Lect. Notes Comput. Sci., vol. 4899, Springer, 2008.

Digital Library

[22]

S. Jha, E. Clarke, C. Langmead, A. Legay, A. Platzer, P. Zuliani, A bayesian approach to model checking biological systems, in: Lect. Notes Comput. Sci., vol. 5688, Springer, 2009.

Digital Library

[23]

K. Kalajdzic, C. Jgourel, A. Lukina, E. Bartocci, A. Legay, S. Smolka, R. Grosu, Feedback control for statistical model checking of cyber-physical systems, in: Lect. Notes Comput. Sci., vol. 9952, Springer, 2016.

[24]

T. Mancini, F. Mari, A. Massini, I. Melatti, F. Merli, E. Tronci, System level formal verification via model checking driven simulation, in: Lect. Notes Comput. Sci., vol. 8044, Springer, 2013.

[25]

T. Mancini, F. Mari, A. Massini, I. Melatti, E. Tronci, Anytime system level verification via random exhaustive hardware in the loop simulation, in: Proc. DSD 2014, IEEE, 2014.

Digital Library

[26]

T. Mancini, F. Mari, A. Massini, I. Melatti, E. Tronci, System level formal verification via distributed multi-core hardware in the loop simulation, in: Proc. PDP 2014, IEEE, 2014.

Digital Library

[27]

T. Mancini, F. Mari, A. Massini, I. Melatti, E. Tronci, SyLVaaS: system level formal verification as a service, in: Proc. PDP 2015, IEEE, 2015.

Digital Library

[28]

T. Mancini, F. Mari, A. Massini, I. Melatti, E. Tronci, Anytime system level verification via parallel random exhaustive hardware in the loop simulation, MicPro, 41 (2016).

Digital Library

[29]

T. Mancini, F. Mari, I. Melatti, I. Salvo, E. Tronci, J. Gruber, B. Hayes, M. Prodanovic, L. Elmegaard, Demand-aware price policy synthesis and verification services for smart grids, in: Proc. SmartGridComm 2014, IEEE, 2014.

[30]

F. Mari, I. Melatti, I. Salvo, E. Tronci, Synthesis of quantized feedback control software for discrete time linear hybrid systems, in: Lect. Notes Comput. Sci., vol. 6174, Springer, 2010.

Digital Library

[31]

F. Mari, I. Melatti, I. Salvo, E. Tronci, Undecidability of quantized state feedback control for discrete time linear hybrid systems, in: Lect. Notes Comput. Sci., vol. 7521, Springer, 2012.

Digital Library

[32]

F. Mari, I. Melatti, I. Salvo, E. Tronci, Model based synthesis of control software from system level formal specifications, ACM Trans. Softw. Eng. Methodol., 23 (2014).

Digital Library

[33]

S. Sankaranarayanan, R. Chang, G. Jiang, F. Ivancic, State space exploration using feedback constraint generation and Monte-Carlo sampling, in: Proc. ACM SIGSOFT 2007, ACM, 2007.

Digital Library

[34]

A. Schrijver, Theory of Linear and Integer Programming, Wiley, 1998.

[35]

H. Sivaraj, G. Gopalakrishnan, Random walk based heuristic algorithms for distributed memory model checking, Electron. Notes Theor. Comput. Sci., 89 (2003).

[36]

L. Thomas, Games, Theory and Applications, Dover, 1980.

[37]

E. Tronci, G. Della Penna, B. Intrigila, M. Venturini Zilli, A probabilistic approach to automatic verification of concurrent systems, in: Proc. APSEC 2001, IEEE, 2001.

Digital Library

[38]

E. Tronci, T. Mancini, I. Salvo, S. Sinisi, F. Mari, I. Melatti, A. Massini, F. Dav, T. Dierkes, R. Ehrig, S. Rblitz, B. Leeners, T.H.C. Krger, M. Egli, F. Ille, Patient-specific models from inter-patient biological models and clinical records, in: Proc. FMCAD 2014, IEEE, 2014.

Digital Library

[39]

G. Verzino, F. Cavaliere, F. Mari, I. Melatti, G. Minei, I. Salvo, Y. Yushtein, E. Tronci, Model checking driven simulation of sat procedures, in: SpaceOps 2012, 2012.

[40]

C.-H. Yang, G. Zhabelova, C.-W. Yang, V. Vyatkin, Cosimulation environment for event-driven distributed controls of smart grid, IEEE Trans. Ind. Inform., 9 (2013).

[41]

P. Zuliani, A. Platzer, E. Clarke, Bayesian statistical model checking with application to Stateflow/Simulink verification, Form. Methods Syst. Des., 43 (2013).

Digital Library

[42]

A. Zutshi, S. Sankaranarayanan, J. Deshmukh, X. Jin, Symbolic-numeric reachability analysis of closed-loop control software, in: Proc. HSCC 2016, ACM, 2016.

Digital Library

Cited By

Picchiami LParmentier MLegay AMancini TTronci E(2025)Scaling up statistical model checking of cyber-physical systems via algorithm ensemble and parallel simulations over HPC infrastructuresJournal of Systems and Software10.1016/j.jss.2024.112238219:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.jss.2024.112238
Mancini TMelatti ITronci E(2023)Optimizing Highly-Parallel Simulation-Based Verification of Cyber-Physical SystemsIEEE Transactions on Software Engineering10.1109/TSE.2023.329843249:9(4443-4455)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3298432
Mancini TMelatti ITronci E(2022)Any-Horizon Uniform Random Sampling and Enumeration of Constrained Scenarios for Simulation-Based Formal VerificationIEEE Transactions on Software Engineering10.1109/TSE.2021.310984248:10(4002-4013)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1109/TSE.2021.3109842
Show More Cited By

Recommendations

Using DEv-PROMELA for Modelling and Verification of Software
SIGSIM-PADS '16: Proceedings of the 2016 ACM SIGSIM Conference on Principles of Advanced Discrete Simulation

Efficient modelling and verification of models need an accurate representation of systems. Especially, PROMELA cannot represent time as quantitative properties. That means some properties depending on time cannot be checked with SPIN model-checker. ...
An integrated environment for HDL verification
IVC '95: Proceedings of the 4th IEEE International Verilog HDL Conference

The functional verification of a digital design is an expensive step in the design process. As designs become more complex, simulation is challenged throughout the design and verification process, both at the low level (implementation verification), to ...
Systematic Formal Verification for Fault-Tolerant Time-Triggered Algorithms

Many critical real-time applications are implemented as time-triggered systems. We present a systematic way to derive such time-triggered implementations from algorithms specified as functional programs (in which form their correctness and fault-...

Comments

Information & Contributors

Information

Published In

cover image Information Processing Letters

Information Processing Letters Volume 122, Issue C

June 2017

31 pages

ISSN:0020-0190

Issue’s Table of Contents

Copyright © Elsevier B.V.

Publisher

Elsevier North-Holland, Inc.

United States

Publication History

Published: 01 June 2017

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Picchiami LParmentier MLegay AMancini TTronci E(2025)Scaling up statistical model checking of cyber-physical systems via algorithm ensemble and parallel simulations over HPC infrastructuresJournal of Systems and Software10.1016/j.jss.2024.112238219:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.jss.2024.112238
Mancini TMelatti ITronci E(2023)Optimizing Highly-Parallel Simulation-Based Verification of Cyber-Physical SystemsIEEE Transactions on Software Engineering10.1109/TSE.2023.329843249:9(4443-4455)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TSE.2023.3298432
Mancini TMelatti ITronci E(2022)Any-Horizon Uniform Random Sampling and Enumeration of Constrained Scenarios for Simulation-Based Formal VerificationIEEE Transactions on Software Engineering10.1109/TSE.2021.310984248:10(4002-4013)Online publication date: 1-Oct-2022
https://dl.acm.org/doi/10.1109/TSE.2021.3109842
Pappagallo A(2021)Effective Analysis of Industry-Relevant Cyber-Physical Systems via Statistical Model CheckingAIxIA 2021 – Advances in Artificial Intelligence10.1007/978-3-031-08421-8_45(655-670)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-031-08421-8_45
Esposito MPicchiami L(2021)A Comparative Study of AI Search Methods for Personalised Cancer Therapy Synthesis in COPASIAIxIA 2021 – Advances in Artificial Intelligence10.1007/978-3-031-08421-8_44(638-654)Online publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-031-08421-8_44
Mancini TMari FMelatti ISalvo ITronci E(2018)An Efficient Algorithm for Network Vulnerability Analysis Under Malicious AttacksFoundations of Intelligent Systems10.1007/978-3-030-01851-1_29(302-312)Online publication date: 29-Oct-2018
https://dl.acm.org/doi/10.1007/978-3-030-01851-1_29

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents