research-article

Do Instance-level Review Diagrams Support Validation Processes of Cyber-Physical System Specifications: Results from a Controlled Experiment

Authors:

Jennifer Brings,

Thorsten WeyerAuthors Info & Claims

ICSSP '20: Proceedings of the International Conference on Software and System Processes

Pages 11 - 20

https://doi.org/10.1145/3379177.3388893

Published: 16 September 2020 Publication History

Abstract

In the field of safety-critical systems, manual reviews are important to ensure high-quality software and to satisfy legal obligations. When applying model-based engineering approaches, no longer are only textual requirements specifications or software code under review, but also model-based specification artifacts like behavioral requirements models. As such behavioral specifications are typically documented on a type-level, errors concerning the interactions between multiple system instances can go unnoticed in manual reviews. This is particularly the case when multiple system instances of the same system type are interacting during runtime, which is typical for cyber-physical systems where networks of cyber-physical systems form dynamically to fulfill an overall purpose. In this paper, we report on a controlled experiment whose results indicate that instance-level review diagrams have -- compared to type-level diagrams - important positive effects on reviewing processes for behavioral specifications of cyber-physical systems. Specifically, the experiment provides empirical evidence that instance-level review diagrams are significantly more expressive and effective than type-level diagrams.

References

[1]

Rajeev Alur, Kousha Etessami, and Mihalis Yannakakis. 2003. Inference of Message Sequence Charts. IEEE Trans. Software Eng. 29, 7 (2003), 623--633. https://doi.org/10.1109/TSE.2003.1214326

Digital Library

[2]

Colin Atkinson, Ralph Gerbig, and Thomas Kühne. 2015. A unifying approach to connections for multi-level modeling. 2015 ACM/IEEE 18th International Conference on Model Driven Engineering Languages and Systems (MODELS) (2015), 216--225. https://doi.org/10.1109/MODELS.2015.7338252

[3]

Alberto Bacchelli and Christian Bird. 2013. Expectations, Outcomes, and Challenges of Modern Code Review. In Proceedings of the 2013 International Conference on Software Engineering (ICSE '13). IEEE Press, 712--721.

[4]

Victor R. Basili, Scott Green, Oliver Laitenberger, Filippo Lanubile, Forrest Shull, Sivert Sørumgård, and Marvin V. Zelkowitz. 1996. The Empirical Investigation of Perspective-Based Reading. Empirical Software Engineering 1, 2 (1996), 133--164. https://doi.org/10.1007/BF00368702

[5]

Tomas Berling and Per Runeson. 2003. Evaluation of a perspective based review method applied in an industrial setting. IEE Proceedings - Software 150, 3 (2003), 177--184. https://doi.org/10.1049/ip-sen:20030483

[6]

Jennifer Brings, Marian Daun, Torsten Bandyszak, Vanessa Stricker, Thorsten Weyer, Elham Mirzaei, Martin Neumann, and Jan Stefan Zernickel. 2019. Model-based documentation of dynamicity constraints for collaborative cyber-physical system architectures: Findings from an industrial case study. J. Syst. Archit. 97 (2019), 153--167. https://doi.org/10.1016/j.sysarc.2019.02.012

Digital Library

[7]

Chien-An Chen, Sara Kalvala, and Jane Sinclair. 2005. Race Conditions in Message Sequence Charts. In Programming Languages and Systems (Lecture Notes in Computer Science), Kwangkeun Yi (Ed.). Springer, Berlin, Heidelberg, 195--211. https://doi.org/10.1007/11575467_14

[8]

Marian Daun, Jennifer Brings, Lisa Krajinski, and Thorsten Weyer. 2019. On the benefits of using dedicated models in validation processes for behavioral specifications. In Proceedings of the International Conference on Software and System Processes, ICSSP 2019, Montreal, QC, Canada, May 25-26, 2019, Stanley M. Sutton Jr, Ove Armbrust, and Regina Hebig (Eds.). IEEE / ACM, 44--53. https://doi.org/10.1109/ICSSP.2019.00016

Digital Library

[9]

Marian Daun, Jennifer Brings, and Thorsten Weyer. 2017. On the Impact of the Model-Based Representation of Inconsistencies to Manual Reviews - Results from a Controlled Experiment. In Conceptual Modeling - 36th International Conference, ER 2017, Valencia, Spain, November 6-9, 2017, Proceedings. 466--473. https://doi.org/10.1007/978-3-319-69904-2_35

[10]

Marian Daun, Jennifer Brings, and Thorsten Weyer. 2018. A semi-automated approach to foster the validation of collaborative networks of cyber-physical systems. In Proceedings of the 4th International Workshop on Software Engineering for Smart Cyber-Physical Systems, ICSE 2018, Gothenburg, Sweden, May 27, 2018. 6--12. https://doi.org/10.1145/3196478.3196483

Digital Library

[11]

Marian Daun, Andrea Salmon, Thorsten Weyer, and Klaus Pohl. 2015. The impact of students' skills and experiences on empirical results: a controlled experiment with undergraduate and graduate students. In Proceedings of the 19th International Conference on Evaluation and Assessment in Software Engineering, EASE 2015, Nanjing, China, April 27-29, 2015, Jian Lv, He Jason Zhang, and Muhammad Ali Babar (Eds.). ACM, Paper 29. https://doi.org/10.1145/2745802.2745829

Digital Library

[12]

Marian Daun, Thorsten Weyer, and Klaus Pohl. 2019. Improving manual reviews in function-centered engineering of embedded systems using a dedicated review model. Software and Systems Modeling 18, 6 (2019), 3421--3459. https://doi.org/10.1007/s10270-019-00723-2

Digital Library

[13]

Claudenir M. Fonseca, João Paulo A. Almeida, Giancarlo Guizzardi, and Victorio Albani de Carvalho. 2018. Multi-level Conceptual Modeling: From a Formal Theory to a Well-Founded Language. In ER.

[14]

Lulu He and Jeffrey C. Carver. 2006. PBR vs. checklist: a replication in the n-fold inspection context. In 2006 International Symposium on Empirical Software Engineering (ISESE 2006), September 21-22, 2006, Rio de Janeiro, Brazil, Guilherme Horta Travassos, José Carlos Maldonado, and Claes Wohlin (Eds.). ACM, 95--104. https://doi.org/10.1145/1159733.1159750

[15]

Frank Houdek and Stefan Schmerler. 2017. Automotive Future and its Impact on Requirements Engineering. In Joint Proceedings of REFSQ-2017 Workshops, Doctoral Symposium, Research Method Track, and Poster Track co-located with the 22nd International Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2017), Essen, Germany, February 27, 2017.

[16]

Loïc Hélouët and Pierre Le Maigat. 2000. Decomposition of Message Sequence Charts. In SAM 2000, 2nd Workshop on SDL and MSC, Col de Porte, Grenoble, France, June 26-28, 2000, Edel Sherratt (Ed.). VERIMAG, IRISA, SDL Forum, 47--60.

[17]

International Telecommunication Union. 2011. Message Sequence Chart (MSC). Technical Report Z120. International Telecommunication Union.

[18]

Andreas Jedlitschka, Marcus Ciolkowski, and Dietmar Pfahl. 2008. Reporting experiments in software engineering. In Guide to Advanced Empirical Software Engineering, Forrest Shull, Janice Singer, and Dag I. K. Sjøberg (Eds.). Springer London, 201--228.

[19]

Mykel J. Kochenderfer, Jessica E. Holland, and James P. Chryssanthacopoulos. 2012. Next generation airborne collision avoidance system. Lincoln Laboratory Journal 19, 1 (2012), 17--33.

[20]

Oliver Laitenberger, Khaled El Emam, and ThomasG. Harbich. 2001. An Internally Replicated Quasi-Experimental Comparison of Checklist and Perspective-Based Reading of Code Documents. IEEE Trans. Software Eng. 27, 5 (2001), 387--421. https://doi.org/10.1109/32.922713

Digital Library

[21]

Emmanuel Letier, Jeff Kramer, Jeff Magee, and Sebastián Uchitel. 2005. Monitoring and control in scenario-based requirements analysis. In 27th International Conference on Software Engineering (ICSE 2005), 15-21 May 2005, St. Louis, Missouri, USA, Gruia-Catalin Roman, William G. Griswold, and Bashar Nuseibeh (Eds.). ACM, 382--391. https://doi.org/10.1145/1062455.1062527

[22]

José Carlos Maldonado, Jeffrey Carver, Forrest Shull, Sandra Camargo Pinto Ferraz Fabbri, Emerson Dória, Luciana Andréia Fondazzi Martimiano, Manoel G. Mendonça, and Victor R. Basili. 2006. Perspective-Based Reading: A Replicated Experiment Focused on Individual Reviewer Effectiveness. Empirical Software Engineering 11, 1 (2006), 119--142. https://doi.org/10.1007/s10664-006-5967-6

Digital Library

[23]

James Miller, Murray Wood, and Marc Roper. 1998. Further Experiences with Scenarios and Checklists. Empirical Software Engineering 3, 1 (1998), 37--64. https://doi.org/10.1023/A:1009735805377

Digital Library

[24]

Bill Mitchell. 2005. Inherent Causal Orderings of Partial Order Scenarios. In Theoretical Aspects of Computing - ICTAC 2004 (Lecture Notes in Computer Science), Zhiming Liu and Keijiro Araki (Eds.). Springer, Berlin, Heidelberg, 113--127. https://doi.org/10.1007/978-3-540-31862-0_10

[25]

Mohammad Moshirpour, Abdolmajid Mousavi, and Behrouz H. Far. 2012. Detecting emergent behavior in distributed systems using scenario-based specifications. International Journal of Software Engineering and Knowledge Engineering 22, 06 (Sept. 2012), 729--746. https://doi.org/10.1142/S0218194012400104

[26]

Adam A. Porter and Lawrence G. Votta. 1998. Comparing Detection Methods For Software Requirements Inspections: A Replication Using Professional Subjects. Empirical Software Engineering 3, 4 (1998), 355--379. https://doi.org/10.1023/A:1009776104355

Digital Library

[27]

Adam A. Porter, Lawrence G. Votta, and Victor R. Basili. 1995. Comparing Detection Methods for Software Requirements Inspections: A Replicated Experiment. IEEE Trans. Software Eng. 21, 6 (1995), 563--575. https://doi.org/10.1109/32.391380

Digital Library

[28]

F. Ricca, M. Di Penta, M. Torchiano, P. Tonella, and M. Ceccato. 2007. The Role of Experience and Ability in Comprehension Tasks Supported by UML Stereotypes. In 29th International Conference on Software Engineering, 2007. ICSE 2007. 375--384. https://doi.org/10.1109/ICSE.2007.86

[29]

Giedre Sabaliauskaite, Shinji Kusumoto, and Katsuro Inoue. 2004. Assessing defect detection performance of interacting teams in object-oriented design inspection. Information & Software Technology 46, 13 (2004), 875--886. https://doi.org/10.1016/j.infsof.2004.03.004

[30]

J.C. Sampaio do Prado Leite and P.A. Freeman. 1991. Requirements validation through viewpoint resolution. IEEE Transactions on Software Engineering 17, 12 (Dec. 1991), 1253--1269. https://doi.org/10.1109/32.106986

[31]

Forrest Shull, Ioana Rus, and Victor R. Basili. 2000. How Perspective-Based Reading Can Improve Requirements Inspections. IEEE Computer 33, 7 (2000), 73--79. https://doi.org/10.1109/2.869376

Digital Library

[32]

Ernst Sikora, Bastian Tenbergen, and Klaus Pohl. 2011. Industry needs and research directions in requirements engineering for embedded systems. Requirements Engineering 17, 1 (Dec. 2011), 57--78. https://doi.org/10.1007/s00766-011-0144-x

[33]

Viktoria Stenkova, Jennifer Brings, Marian Daun, and Thorsten Weyer. 2019. Generic Negative Scenarios for the Specification of Collaborative Cyber-Physical Systems. In Conceptual Modeling - 38th International Conference, ER 2019, Salvador, Brazil, November 4-7, 2019, Proceedings. 412--419. https://doi.org/10.1007/978-3-030-33223-5_34

[34]

Sebastian Uchitel, Jeff Kramer, and Jeff Magee. 2001. Detecting Implied Scenarios in Message Sequence Chart Specifications. In Proceedings of the 8th European Software Engineering Conference Held Jointly with 9th ACM SIGSOFT International Symposium on Foundations of Software Engineering (ESEC/FSE-9). Association for Computing Machinery, New York, NY, USA, 74--82. https://doi.org/10.1145/503209.503220

[35]

S. Vegas, C. Apa, and N. Juristo. 2016. Crossover Designs in Software Engineering Experiments: Benefits and Perils. IEEE Transactions on Software Engineering 42, 2 (Feb. 2016), 120--135. https://doi.org/10.1109/TSE.2015.2467378

Digital Library

[36]

Viswanath Venkatesh and Hillol Bala. 2008. Technology Acceptance Model 3 and a Research Agenda on Interventions. Decision Sciences 39, 2 (May 2008), 273--315. https://doi.org/10.1111/j.1540--5915.2008.00192.x

[37]

M. Weber and J. Weisbrod. 2002. Requirements engineering in automotive development-experiences and challenges. In IEEE Joint International Conference on Requirements Engineering, 2002. Proceedings. 331--340. https://doi.org/10.1109/ICRE.2002.1048546

[38]

Claes Wohlin. 2000. Experimentation in software engineering: An introduction. Kluwer Academic, Boston, Mass.

Cited By

Daun MBrings J(2023)Investigating Factors Influencing Students’ Assessment of Conceptual ModelsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593960(470-474)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593960
Daun MBrings J(2023)Aggregating N-fold Requirements Inspection ResultsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593465(339-347)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593465
Tell PRaffo DHuang LSteinmacher IBritto RTüzün EClarke P(2021)Summary of the 1st ICSSP-ICGSE Joint EventACM SIGSOFT Software Engineering Notes10.1145/3417564.341757645:4(31-34)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3417564.3417576
Show More Cited By

Index Terms

Do Instance-level Review Diagrams Support Validation Processes of Cyber-Physical System Specifications: Results from a Controlled Experiment

Recommendations

Model inspections in the engineering of collaborative cyber‐physical systems with instance‐level review diagrams
Abstract
Model inspections are important to ensure high‐quality software and to satisfy legal obligations in model‐based engineering processes. As model‐based specifications are typically documented on type‐level, errors concerning the interactions between ...

This paper contributes a definition of instance‐level review diagrams in message sequence charts notation and reports results of a controlled experiment to evaluate instance‐level review diagrams. Results show that instance‐level review models support the ...
A semi-automated approach to foster the validation of collaborative networks of cyber-physical systems
SEsCPS '18: Proceedings of the 4th International Workshop on Software Engineering for Smart Cyber-Physical Systems

Cyber-physical systems form collaborative networks dynamically at runtime. In the collaboration of multiple systems, behavior emerges in the interplay of the collaborating instances. This emergent behavior raises challenges for the validation of cyber-...
Modeling and debugging numerical constraints of cyber-physical systems design
SoICT '13: Proceedings of the 4th Symposium on Information and Communication Technology

To design and analyze Cyber-Physical Systems (CPSs), engineers should consider computation and physical processes at the same time. Engineers require a design description which is easy to understand. SysML provides diagrammatic notations which are ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSSP '20: Proceedings of the International Conference on Software and System Processes

June 2020

208 pages

ISBN:9781450375122

DOI:10.1145/3379177

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 September 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Bundesministerium für Bildung und Forschung

Conference

ICSSP '20

Sponsor:

SIGSOFT

ICSSP '20: International Conference on Software and System Processes

June 26 - 28, 2020

Seoul, Republic of Korea

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
76
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Daun MBrings J(2023)Investigating Factors Influencing Students’ Assessment of Conceptual ModelsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593960(470-474)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593960
Daun MBrings J(2023)Aggregating N-fold Requirements Inspection ResultsProceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering10.1145/3593434.3593465(339-347)Online publication date: 14-Jun-2023
https://dl.acm.org/doi/10.1145/3593434.3593465
Tell PRaffo DHuang LSteinmacher IBritto RTüzün EClarke P(2021)Summary of the 1st ICSSP-ICGSE Joint EventACM SIGSOFT Software Engineering Notes10.1145/3417564.341757645:4(31-34)Online publication date: 18-Oct-2021
https://dl.acm.org/doi/10.1145/3417564.3417576
Daun MBrings JObe PStenkova V(2021)Reliability of self-rated experience and confidence as predictors for students’ performance in software engineeringEmpirical Software Engineering10.1007/s10664-021-09972-626:4Online publication date: 9-Jun-2021
https://doi.org/10.1007/s10664-021-09972-6
Daun MBrings JKrajinski LStenkova VBandyszak T(2021)A GRL-compliant iStar extension for collaborative cyber-physical systemsRequirements Engineering10.1007/s00766-021-00347-3Online publication date: 4-Feb-2021
https://doi.org/10.1007/s00766-021-00347-3
Daun MBrings JWeyer T(2021)Model inspections in the engineering of collaborative cyber‐physical systems with instance‐level review diagramsJournal of Software: Evolution and Process10.1002/smr.2392Online publication date: 17-Oct-2021
https://doi.org/10.1002/smr.2392

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents