Article

A concept analysis inspired greedy algorithm for test suite minimization

Authors:

Sriraman Tallam,

Neelam GuptaAuthors Info & Claims

PASTE '05: Proceedings of the 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering

Pages 35 - 42

https://doi.org/10.1145/1108792.1108802

Published: 05 September 2005 Publication History

Abstract

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible and to ensure that changes to existing software do not break the software. Test suites once developed are reused and updated frequently as the software evolves. As a result, some test cases in the test suite may become redundant as the software is modified over time since the requirements covered by them are also covered by other test cases. Due to the resource and time constraints for re-executing large test suites, it is important to develop techniques to minimize available test suites by removing redundant test cases. In general, the test suite minimization problem is NP complete. In this paper, we present a new greedy heuristic algorithm for selecting a minimal subset of a test suite T that covers all the requirements covered by T. We show how our algorithm was inspired by the concept analysis framework. We conducted experiments to measure the extent of test suite reduction obtained by our algorithm and prior heuristics for test suite minimization. In our experiments, our algorithm always selected same size or smaller size test suite than that selected by prior heuristics and had comparable time performance.

References

[1]

H. Agrawal, "Dominators, super blocks, and program coverage," 21st ACM SIGPLAN-SIGACT symposium on Principles of Programming Languages, Portland, Oregon, 1994.

Digital Library

[2]

H. Agrawal, "Efficient Coverage Testing Using Global Dominator Graphs," 1999 ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering, Toulouse, France, 1999.

Digital Library

[3]

G. Birkhoff Lattice Theory, volume 5, American Mathematical Soc. Colloquium Publications, 1940.

[4]

J. Black, E. Melachrinoudis and D. Kaeli, "Bi-Criteria Models for All-Uses Test Suite Reduction," 26th International Conference on Software Engineering, Edinburgh, Scotland, UK, 2004

Digital Library

[5]

V. Chvatal. "A Greedy Heuristic for the Set-Covering Problem." Mathematics of Operations Research. 4(3), August 1979.

[6]

T. H. Cormen, C. E. Leiserson, R. L. Rivest and C. Stein "Introduction to Algorithms", MIT Press, Second Edition, September 2001.

Digital Library

[7]

M. R. Garey and D. S. Johnson, "Computers and Intractability-A Guide to the Theory of NP-Completeness," V Klee, Ed. Freeman, New York, 1979.

Digital Library

[8]

M. J. Harrold, R. Gupta and M. L. Soffa, "A Methodology for Controlling the Size of a Test Suite," ACM Transactions on Software Engineering and Methodology, 2(3):270--285, July 1993.

Digital Library

[9]

J. R. Horgan and S. A. London, "ATAC: A data flow coverage testing tool for C," in Proceedings of Symposium on Assessment of Quality Software Development Tools, pages 2--10, May 1992.

[10]

M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments on the Effectiveness of Dataflow- and Controlflow-based Test Adequacy Criteria," 16th International Conference on Software Engineering, May 1994.

Digital Library

[11]

http://www.cse.unl.edu/~galileo/sir

[12]

J. A. Jones and M. J. Harrold, "Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage," IEEE Transactions on Software Engineering, 29(3):195--209, March 2003.

Digital Library

[13]

M. Marre and A. Bertolino, "Using Spanning Sets for Coverage Testing," IEEE Transactions on Software Engineering, 29(11):974--984, Nov. 2003.

Digital Library

[14]

G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong, "An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites," International Conference on Software Maintenance, November 1998.

Digital Library

[15]

S. Sampath, V. Mihaylov, A. Souter and L. Pollock "A Scalable Approach to User-Session based Testing of Web Applications through Concept Analysis," in proceedings of Automated Software Engineering, 19th International Conference on (ASE'04) Linz, Austria, September 2004,

Digital Library

[16]

S. Sprenkle, S. Sampath, E. Gibson, A. Souter, L. Pollock, "An Empirical Comparison of Test Suite Reduction Techniques for User-session-based Testing of Web Applications," Technical Report 2005--009, Computer and Information Sciences, University of Delaware, November 2004

[17]

"The LLVM Compiler Infrastructure Project," http://llvm.cs.uiuc.edu/

[18]

W. E. Wong, J. R. Horgan, S. London, and A. P. Mathur. "Effect of Test Set Minimization on Fault Detection Effectiveness." Software Practice and Experience. 28(4):347--369, April 1998.

Digital Library

Cited By

Asgari Araghi MRafe VKhendek F(2024)Using data mining techniques to generate test cases from graph transformation systems specificationsAutomated Software Engineering10.1007/s10515-024-00417-031:1Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.1007/s10515-024-00417-0
Koitz-Hristov RStracke LWotawa FGarrido AWong WDe Angelis GDo HNguyen B(2022)Checked coverage for test suite reductionProceedings of the 3rd ACM/IEEE International Conference on Automation of Software Test10.1145/3524481.3527216(6-16)Online publication date: 17-May-2022
https://dl.acm.org/doi/10.1145/3524481.3527216
A MTotla TRenwa YR CSubbiah SDharmaraj TPrakash S(2022)Comparative Analysis of Ant Colony Optimization and Particle Swarm Optimization for Test Case Prioritization2022 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT)10.1109/3ICT56508.2022.9990713(680-686)Online publication date: 20-Nov-2022
https://doi.org/10.1109/3ICT56508.2022.9990713
Show More Cited By

Index Terms

A concept analysis inspired greedy algorithm for test suite minimization
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

A concept analysis inspired greedy algorithm for test suite minimization

Software testing and retesting occurs continuously during the software development lifecycle to detect errors as early as possible and to ensure that changes to existing software do not break the software. Test suites once developed are reused and ...
Regression testing minimization, selection and prioritization: a survey

Regression testing is a testing activity that is performed to provide confidence that changes do not harm the existing behaviour of the software. Test suites tend to grow in size as software evolves, often making it too costly to execute entire test ...
Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction

Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PASTE '05: Proceedings of the 6th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering

September 2005

118 pages

ISBN:1595932399

DOI:10.1145/1108792

Editors:
Michael Ernst
MIT
,
Thomas Jensen
CNRS

ACM SIGSOFT Software Engineering Notes Volume 31, Issue 1
January 2006
203 pages
ISSN:0163-5948
DOI:10.1145/1108768
Issue’s Table of Contents

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 September 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

PASTE05

Sponsor:

PASTE05: PASTE '05 - ACM SIGPLAN-SIGSOFT Workshop on Program Analysis for Software Tools and Engineering

September 5 - 6, 2005

Lisbon, Portugal

Acceptance Rates

Overall Acceptance Rate 57 of 159 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

171
Total Citations
View Citations
1,013
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)1

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Asgari Araghi MRafe VKhendek F(2024)Using data mining techniques to generate test cases from graph transformation systems specificationsAutomated Software Engineering10.1007/s10515-024-00417-031:1Online publication date: 21-Feb-2024
https://dl.acm.org/doi/10.1007/s10515-024-00417-0
Koitz-Hristov RStracke LWotawa FGarrido AWong WDe Angelis GDo HNguyen B(2022)Checked coverage for test suite reductionProceedings of the 3rd ACM/IEEE International Conference on Automation of Software Test10.1145/3524481.3527216(6-16)Online publication date: 17-May-2022
https://dl.acm.org/doi/10.1145/3524481.3527216
A MTotla TRenwa YR CSubbiah SDharmaraj TPrakash S(2022)Comparative Analysis of Ant Colony Optimization and Particle Swarm Optimization for Test Case Prioritization2022 International Conference on Innovation and Intelligence for Informatics, Computing, and Technologies (3ICT)10.1109/3ICT56508.2022.9990713(680-686)Online publication date: 20-Nov-2022
https://doi.org/10.1109/3ICT56508.2022.9990713
Hussein HYounes AAbdelmoez W(2020)Quantum-Inspired Genetic Algorithm for Solving the Test Suite Minimization ProblemWSEAS TRANSACTIONS ON COMPUTERS10.37394/23205.2020.19.2019(143-155)Online publication date: 17-Aug-2020
https://doi.org/10.37394/23205.2020.19.20
Xue YLi Y(2020)Multi-objective Integer Programming Approaches for Solving the Multi-criteria Test-suite Minimization ProblemACM Transactions on Software Engineering and Methodology10.1145/339203129:3(1-50)Online publication date: 1-Jun-2020
https://dl.acm.org/doi/10.1145/3392031
Alsharif AKapfhammer GMcMinn P(2020)Hybrid Methods for Reducing Database Schema Test SuitesProceedings of the IEEE/ACM 1st International Conference on Automation of Software Test10.1145/3387903.3389305(41-50)Online publication date: 7-Oct-2020
https://dl.acm.org/doi/10.1145/3387903.3389305
Coviello CRomano SScanniello GAntoniol G(2020)GASSERProceedings of the 14th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM)10.1145/3382494.3422157(1-6)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3382494.3422157
Lau JSivaraman AZhang QGulzar MCong JKim MRothermel GBae D(2020)HeteroRefactorProceedings of the ACM/IEEE 42nd International Conference on Software Engineering10.1145/3377811.3380340(493-505)Online publication date: 27-Jun-2020
https://dl.acm.org/doi/10.1145/3377811.3380340
Alsharif AKapfhammer GMcMinn P(2020)STICCER: Fast and Effective Database Test Suite Reduction Through Merging of Similar Test Cases2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST)10.1109/ICST46399.2020.00031(220-230)Online publication date: Oct-2020
https://doi.org/10.1109/ICST46399.2020.00031
Zhang ZWang YWang ZQian J(2019)How to Effectively Reduce Tens of Millions of Tests: An Industrial Case Study on Adaptive Random TestingIEEE Transactions on Reliability10.1109/TR.2019.292764368:4(1429-1443)Online publication date: Dec-2019
https://doi.org/10.1109/TR.2019.2927643
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents