research-article

MOTSD: a multi-objective test selection tool using test suite diagnosability

Authors:

Daniel Correia,

Rui Abreu,

Pedro Santos,

João NadkarniAuthors Info & Claims

ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 1070 - 1074

https://doi.org/10.1145/3338906.3341187

Published: 12 August 2019 Publication History

Get Access

Abstract

Performing regression testing on large software systems becomes unfeasible as it takes too long to run all the test cases every time a change is made. The main motivation of this work was to provide a faster and earlier feedback loop to the developers at OutSystems when a change is made. The developed tool, MOTSD, implements a multi-objective test selection approach in a C# code base using a test suite diagnosability metric and historical metrics as objectives and it is powered by a particle swarm optimization algorithm. We present implementation challenges, current experimental results and limitations of the tool when applied in an industrial context. Screencast demo link: <a>https://www.youtube.com/watch?v=CYMfQTUu2BE</a>

References

[1]

Carlos A Coello Coello, Gary B Lamont, and David A Van Veldhuizen. 2007. Evolutionary Algorithms for Solving Multi-Objective Problems. Vol. 5. Springer.

Digital Library

Google Scholar

[2]

J. Kennedy and R. C. Eberhart. 1997. A Discrete Binary Version of the Particle Swarm Algorithm. In 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation, Vol. 5. 4104–4108 vol.5.

Google Scholar

[3]

Mateusz Machalica, Alex Samylkin, Meredith Porth, and Satish Chandra. 2018. Predictive Test Selection. CoRR abs/1810.05286 (2018). arXiv: 1810.05286 http: //arxiv.org/abs/1810.05286

Google Scholar

[4]

Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Evaluation in information retrieval. In Introduction to Information Retrieval. Cambridge University Press, Chapter 8, 151–175.

Google Scholar

[5]

Atif Memon, Zebao Gao, Bao Nguyen, Sanjeev Dhanda, Eric Nickell, Rob Siemborski, and John Micco. 2017. Taming Google-scale Continuous Testing. In Proceedings of the 39th International Conference on Software Engineering: Software Engineering in Practice Track (ICSE-SEIP ’17). IEEE Press, Piscataway, NJ, USA, 233–242.

Digital Library

Google Scholar

[6]

Alexandre Perez, Rui Abreu, and Arie van Deursen. 2017. A Test-suite Diagnosability Metric for Spectrum-based Fault Localization Approaches. In Proceedings of the 39th International Conference on Software Engineering (ICSE ’17). IEEE Press, Piscataway, NJ, USA, 654–664.

Digital Library

Google Scholar

[7]

Shin Yoo and Mark Harman. 2012. Regression testing minimization, selection and prioritization: a survey. Software Testing, Verification and Reliability 22, 2 (2012), 67–120.

Crossref

Google Scholar

[8]

Aimin Zhou, Bo-Yang Qu, Hui Li, Shi-Zheng Zhao, Ponnuthurai Nagaratnam Suganthan, and Qingfu Zhang. 2011. Multiobjective evolutionary algorithms: A survey of the state of the art. Swarm and Evolutionary Computation 1, 1 (2011), 32 – 49. Abstract 1 Introduction and Motivation 2 Features 3 Implementation 4 Results 5 Conclusions Acknowledgments References

Crossref

Google Scholar

Cited By

View all

Greca RMiranda BBertolino A(2023)State of Practical Applicability of Regression Testing Research: A Live Systematic Literature ReviewACM Computing Surveys10.1145/357985155:13s(1-36)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3579851
Li YWang ZWang JChen JMou RLi G(2023)Semantic‐aware two‐phase test case prioritization for continuous integrationSoftware Testing, Verification and Reliability10.1002/stvr.186434:1Online publication date: 26-Sep-2023
https://doi.org/10.1002/stvr.1864
Gupta NSharma APachariya M(2022)Multi-objective test suite optimization for detection and localization of software faultsJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2020.01.00934:6(2897-2909)Online publication date: Jun-2022
https://doi.org/10.1016/j.jksuci.2020.01.009
Show More Cited By

Index Terms

MOTSD: a multi-objective test selection tool using test suite diagnosability
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Software maintenance tools

Recommendations

An industrial application of test selection using test suite diagnosability
ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Performing full regression testing every time a change is made on large software systems tends to be unfeasible as it takes too long to run all the test cases. The main motivation of this work was to provide a shorter and earlier feedback loop to the ...
A test-suite diagnosability metric for spectrum-based fault localization approaches
ICSE '17: Proceedings of the 39th International Conference on Software Engineering

Current metrics for assessing the adequacy of a test-suite plainly focus on the number of components (be it lines, branches, paths) covered by the suite, but do not explicitly check how the tests actually exercise these components and whether they ...
Symbolic Model-based Test Selection

This paper addresses the problem of model-based off-line selection of test cases for testing the conformance of a black-box implementation with respect to a specification, in the context of reactive systems. Efficient solutions to this problem have been ...

Comments

Information & Contributors

Information

Published In

ESEC/FSE 2019: Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 2019

1264 pages

ISBN:9781450355728

DOI:10.1145/3338906

General Chairs:
Marlon Dumas
University of Tartu, Estonia
,
Dietmar Pfahl
University of Tartu, Estonia
,
Program Chairs:
Sven Apel
Saarland University, Germany
,
Alessandra Russo
Imperial College, UK

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 August 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ESEC/FSE '19

Sponsor:

SIGSOFT

ESEC/FSE '19: 27th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

August 26 - 30, 2019

Tallinn, Estonia

Acceptance Rates

Overall Acceptance Rate 112 of 543 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
228
Total Downloads

Downloads (Last 12 months)6
Downloads (Last 6 weeks)3

Reflects downloads up to 30 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Greca RMiranda BBertolino A(2023)State of Practical Applicability of Regression Testing Research: A Live Systematic Literature ReviewACM Computing Surveys10.1145/357985155:13s(1-36)Online publication date: 13-Jul-2023
https://dl.acm.org/doi/10.1145/3579851
Li YWang ZWang JChen JMou RLi G(2023)Semantic‐aware two‐phase test case prioritization for continuous integrationSoftware Testing, Verification and Reliability10.1002/stvr.186434:1Online publication date: 26-Sep-2023
https://doi.org/10.1002/stvr.1864
Gupta NSharma APachariya M(2022)Multi-objective test suite optimization for detection and localization of software faultsJournal of King Saud University - Computer and Information Sciences10.1016/j.jksuci.2020.01.00934:6(2897-2909)Online publication date: Jun-2022
https://doi.org/10.1016/j.jksuci.2020.01.009
Verma AChoudhary ATiwari S(2022)A novel chaotic archimedes optimization algorithm and its application for efficient selection of regression test casesInternational Journal of Information Technology10.1007/s41870-022-01031-715:2(1055-1068)Online publication date: 25-Jul-2022
https://doi.org/10.1007/s41870-022-01031-7
Dalal RKhari MBhal TSharma K(2022)NGA-II-Based Test Suite Minimization in SoftwareOptimization of Automated Software Testing Using Meta-Heuristic Techniques10.1007/978-3-031-07297-0_1(1-11)Online publication date: 27-Sep-2022
https://doi.org/10.1007/978-3-031-07297-0_1
Bajaj ASangwan O(2021)Discrete cuckoo search algorithms for test case prioritizationApplied Soft Computing10.1016/j.asoc.2021.107584(107584)Online publication date: Jun-2021
https://doi.org/10.1016/j.asoc.2021.107584
Verma AChoudhary ATiwari S(2021)Regression Test Case Selection: A Comparative Analysis of Metaheuristic AlgorithmsProceedings of Second Doctoral Symposium on Computational Intelligence10.1007/978-981-16-3346-1_49(605-615)Online publication date: 20-Sep-2021
https://doi.org/10.1007/978-981-16-3346-1_49
Landing CTahvili SHaggren HLangkvis MMuhammad ALoufi A(2020)Cluster-Based Parallel Testing Using Semantic Analysis2020 IEEE International Conference On Artificial Intelligence Testing (AITest)10.1109/AITEST49225.2020.00022(99-106)Online publication date: Aug-2020
https://doi.org/10.1109/AITEST49225.2020.00022
Gupta NSharma APachariya M(2020)Testing and debugging: an empirical evaluation of integrated approachesSādhanā10.1007/s12046-020-01382-645:1Online publication date: 6-Jun-2020
https://doi.org/10.1007/s12046-020-01382-6

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Abstract

References

Cited By

Index Terms

Recommendations

An industrial application of test selection using test suite diagnosability

A test-suite diagnosability metric for spectrum-based fault localization approaches

Symbolic Model-based Test Selection

Comments

Information

Published In

Sponsors

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Share

Share this Publication link

Share on social media

Affiliations