An experimental analysis of fault detection capabilities of covering array constructors
Pages 246 - 247
Abstract
Combinatorial Interaction Testing (CIT) aims at constructing an effective test suite, such as a Covering Array (CA), that can detect faults that are caused by the interaction of parameters. In this paper, we report on some empirical studies conducted to examine the fault detection capabilities of five popular CA constructors: ACTS, Jenny, PICT, CASA, and TCA. The experimental results indicate that Jenny has the best performance, because it achieves better fault detection than the other four constructors in many cases. Our results also indicate that CAs generated using ACTS, PICT, or CASA should be prioritized before testing.
References
[1]
Jenny: A freely available CA constructor. http://burtleburtle.net/bob/math/jenny.html.
[2]
J. Czerwonka. 2006. Pairwise Testing in Real World: Practical Extensions to Test Case Generators. In Proceedings of the 24th Pacific Northwest Software Quality Conference (PNSQC'06). 419--430.
[3]
B. J. Garvin, M. B. Cohen, and M. B. Dwyer. 2011. Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Software Engineering 16, 1 (2011), 61--102.
[4]
Y. Lei, R. Kacker, D. R. Kuhn, and V. Okun. 2008. IPOG/IPOD: Efficient Test Generation for Multi-Way Software Testing. Software Testing, Verification, and Reliability 18, 3 (2008), 125--148.
[5]
J. Lin, C. Luo, S. Cai, K. Su, D. Hao, and L. Zhang. 2015. TCA: An Efficient Two-Mode Meta-Heuristic Algorithm for Combinatorial Test Generation. In Proceedings of the 30th IEEE/ACM International Conference on Automated Software Engineering (ASE'15). 494--505.
[6]
C. Nie and H. Leung. 2011. A survey of combinatorial testing. ACM Computer Survey 43, 2 (2011), 11:1--11:29.
[7]
G. Rothermel, R. H. Untch, Chengyun Chu, and M. J. Harrold. 2001. Prioritizing Test Cases for Regression Testing. IEEE Transactions on Software Engineering 27, 10 (2001), 929--948.
Index Terms
- An experimental analysis of fault detection capabilities of covering array constructors
Recommendations
Prioritizing Variable-Strength Covering Array
COMPSAC '13: Proceedings of the 2013 IEEE 37th Annual Computer Software and Applications ConferenceCombinatorial interaction testing is a well-studied testing strategy, and has been widely applied in practice. Combinatorial interaction test suite, such as fixed-strength and variable-strength interaction test suite, is widely used for combinatorial ...
Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction
Software testing is a critical part of software development. As new test cases are generated over time due to software modifications, test suite sizes may grow significantly. Because of time and resource constraints for testing, test suite minimization ...
Comparing the Fault Detection Effectiveness of N-way and Random Test Suites
ISESE '04: Proceedings of the 2004 International Symposium on Empirical Software EngineeringSoftware testing plays a critical role in the timely delivery of high-quality software systems. Despite the important role that testing plays, little is known about the fault detection effectiveness of many testing techniques. In this paper we ...
Comments
Information & Contributors
Information
Published In
May 2018
231 pages
ISBN:9781450356633
DOI:10.1145/3183440
- Conference Chair:
- Michel Chaudron,
- General Chair:
- Ivica Crnkovic,
- Program Chairs:
- Marsha Chechik,
- Mark Harman
Copyright © 2018 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
- SIGSOFT: ACM Special Interest Group on Software Engineering
- IEEE-CS: Computer Society
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 May 2018
Check for updates
Author Tags
Qualifiers
- Poster
Conference
ICSE '18
Sponsor:
- SIGSOFT
- IEEE-CS
ICSE '18: 40th International Conference on Software Engineering
May 27 - June 3, 2018
Gothenburg, Sweden
Acceptance Rates
Overall Acceptance Rate 276 of 1,856 submissions, 15%
Upcoming Conference
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 103Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 23 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in