research-article

Effective Lightweight Software Fault Localization based on Test Suite Optimization

Authors:

Roheet Bhatnagar,

Devesh Kumar SrivastavaAuthors Info & Claims

ISEC '22: Proceedings of the 15th Innovations in Software Engineering Conference

Article No.: 9, Pages 1 - 10

https://doi.org/10.1145/3511430.3511437

Published: 24 February 2022 Publication History

Abstract

Automated software fault localization techniques aid developers in program debugging by identifying the probable locations of faults in a program with minimum human intervention. As software is growing in complexity and scale today, increasing the efficiency of fault localization techniques is very essential in order to reduce the overall software development cost. The effectiveness of the test suites used in the fault localization process has a significant impact on the efficiency of the process. Previous studies, on the other hand, have placed less focus on the adequacy of test suites for the fault localization process. We apply optimized test suites in this paper to improve the performance of software fault localization in a single-fault scenario. For our experiments, we use spectrum-based fault localization (SBFL) techniques. Because of its minimal computing overhead and scalability, spectrum-based fault localization is a popular, efficient, and yet lightweight fault localization technique. To optimize the test suite, we employ a heuristic that asserts that if a faulty statement is executed by a passing test case, that test case will have a negative impact on fault localization performance. In contrast, if a passing test case does not execute the faulty statement, the faulty statement's suspiciousness increases, which has a positive impact on fault localization performance. The test suite optimization approach used in this paper significantly improves fault localization performance, as demonstrated by our experiments. The results show that the proposed method efficiently reduces the number of statements examined by about 84.94 percent on average.

References

[1]

W. Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A Survey on Software Fault Localization. IEEE Transactions on Software Engineering 42 (8): 707–40. https://doi.org/10.1109/tse.2016.2521368.

Digital Library

[2]

Higor A. de Souza, Marcos L. Chaim, and Fabio Kon. 2016. Spectrum-Based Software Fault Localization: A Survey of Techniques, Advances, and Challenges. ArXiv [Cs.SE]. http://arxiv.org/abs/1607.04347.

[3]

Amol Saxena, Roheet Bhatnagar, and Devesh Kumar Srivastava. 2021. Improving Effectiveness of Spectrum-Based Software Fault Localization Using Mutation Testing. In 2021 2nd International Conference for Emerging Technology (INCET). IEEE. https://doi.org/10.1109/incet51464.2021.9456109.

[4]

Yan Lei, Chengnian Sun, Xiaoguang Mao, and Zhendong Su. 2018. How Test Suites Impact Fault Localisation Starting from the Size. IET Software 12 (3): 190–205. https://doi.org/10.1049/iet-sen.2017.0026.

Digital Library

[5]

Alexandre Perez, Rui Abreu, and Arie van Deursen. 2017. “A Test-Suite Diagnosability Metric for Spectrum-Based Fault Localization Approaches.” In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE. https://doi.org/10.1109/icse.2017.66.

Digital Library

[6]

Laura Inozemtseva, and Reid Holmes. 2014. “Coverage Is Not Strongly Correlated with Test Suite Effectiveness.” In Proceedings of the 36th International Conference on Software Engineering. New York, NY, USA: ACM. https://doi.org/10.1145/2568225.2568271.

Digital Library

[7]

Jifeng Xuan, and Martin Monperrus. 2014. “Test Case Purification for Improving Fault Localization.” In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. New York, NY, USA: ACM. https://doi.org/10.1145/2635868.2635906.

Digital Library

[8]

Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2007. On the Accuracy of Spectrum-Based Fault Localization. In Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007). IEEE. https://doi.org/10.1109/taicpart.2007.4344104.

[9]

Yong Wang, Zhiqiu Huang, Yong Li, and Bingwu Fang. 2017. Lightweight Fault Localization Combined with Fault Context to Improve Fault Absolute Rank. Science China Information Sciences 60 (9). https://doi.org/10.1007/s11432-017-9112-2.

[10]

Spencer Pearson, Jose Campos, Rene Just, Gordon Fraser, Rui Abreu, Michael D. Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and Improving Fault Localization. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE. https://doi.org/10.1109/icse.2017.62.

Digital Library

[11]

James A. Jones, and Mary Jean Harrold. 2005. Empirical Evaluation of the Tarantula Automatic Fault-Localization Technique. In Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering - ASE ’05. New York, New York, USA: ACM Press. https://doi.org/10.1145/1101908.1101949.

Digital Library

[12]

Rui Abreu, Peter Zoeteweij, and Arjan Van Gemund. 2006. An Evaluation of Similarity Coefficients for Software Fault Localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06). IEEE. https://doi.org/10.1109/prdc.2006.18.

Digital Library

[13]

Rui Abreu, Peter Zoeteweij, Arjan J.C.van Gemund. 2011. Simultaneous Debugging of Software Faults. The Journal of Systems and Software 84 (4): 573–86. https://doi.org/10.1016/j.jss.2010.11.915.

Digital Library

[14]

W. Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. 2014. The DStar Method for Effective Software Fault Localization. IEEE Transactions on Reliability 63 (1): 290–308. https://doi.org/10.1109/tr.2013.2285319.

[15]

Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan J. C. van Gemund. 2009. A Practical Evaluation of Spectrum-Based Fault Localization. The Journal of Systems and Software 82 (11): 1780–92. https://doi.org/10.1016/j.jss.2009.06.035.

Digital Library

[16]

Hyunsook Do, Sebastian Elbaum, and Gregg Rothermel. 2005. Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and Its Potential Impact. Empirical Software Engineer 10 (4): 405–35. https://doi.org/10.1007/s10664-005-3861-2.

Digital Library

[17]

Yanbing Yu, James A. Jones, and Mary Jean Harrold. 2008. An Empirical Study of the Effects of Test-Suite Reduction on Fault Localization. In Proceedings of the 13th International Conference on Software Engineering - ICSE ’08. New York, New York, USA: ACM Press. https://doi.org/10.1145/1368088.1368116.

Digital Library

[18]

Chris Parnin, and Alessandro Orso. 2011. Are Automated Debugging Techniques Actually Helping Programmers? In Proceedings of the 2011 International Symposium on Software Testing and Analysis - ISSTA ’11. New York, New York, USA: ACM Press. https://doi.org/10.1145/2001420.2001445.

Digital Library

[19]

James A. Jones, Mary Jean Harrold, and John Stasko. 2002. “Visualization of Test Information to Assist Fault Localization.” In Proceedings of the 24th International Conference on Software Engineering - ICSE ’02. New York, New York, USA: ACM Press. https://doi.org/10.1145/581339.581397.

Digital Library

[20]

Wes Masri, and Rawad Abou Assi. 2014. “Prevalence of Coincidental Correctness and Mitigation of Its Impact on Fault Localization.” ACM Transactions on Software Engineering and Methodology 23 (1): 1–28. https://doi.org/10.1145/2559932.

Digital Library

[21]

Yong Wang, Zhiqiu Huang, Rongcun Wang, and Qiao Yu. 2018. “Spectrum-Based Fault Localization Using Fault Triggering Model to Refine Fault Ranking List.” IEICE Transactions on Information and Systems E101.D (10): 2436–46. https://doi.org/10.1587/transinf.2017edp7386.

[22]

Shounak Roychowdhury, and Sarfraz Khurshid. 2011. “Software Fault Localization Using Feature Selection.” In Proceedings of the International Workshop on Machine Learning Technologies in Software Engineering - MALETS ’11. New York, New York, USA: ACM Press. https://doi.org/10.1145/2070821.2070823.

Digital Library

[23]

Ting Shu, Lei Wang, and Jinsong Xia. 2017. “Fault Localization Using a Failed Execution Slice.” In 2017 International Conference on Software Analysis, Testing and Evolution (SATE). IEEE. https://doi.org/10.1109/sate.2017.13.

[24]

Jeongho Kim, Jonghee Park and Eunseok Lee. 2016. “A New Spectrum-based Fault Localization With the Technique of Test Case Optimization.” J. Inf. Sci. Eng. 32 (2016): 177-196.

[25]

Horvath, Ferenc, Victor Schnepper Lacerda, Arpad Beszedes, Laszlo Vidacs, and Tibor Gyimothy. 2019. “A New Interactive Fault Localization Method with Context Aware User Feedback.” In 2019 IEEE 1st International Workshop on Intelligent Bug Fixing (IBF). IEEE. https://doi.org/10.1109/ibf.2019.8665415.

[26]

A. Zakari, S. P. Lee, and I. A. T. Hashem, “A single fault localization technique based on failed test input,” Array, vol. 3–4, no. 100008, p. 100008, 2019.

[27]

D. Ghosh and J. Singh, “Spectrum-based multi-fault localization using Chaotic Genetic Algorithm,” Inf. Softw. Technol., vol. 133, no. 106512, p. 106512, 2021.

[28]

Y. Li, S. Wang, and T. Nguyen, “Fault localization with code coverage representation learning,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE), 2021.

Digital Library

Cited By

Wu JZhang ZYang DXu JHe JMao X(2024)Time-Aware Spectrum-Based Bug Localization for Hardware Design Code with Data PurificationACM Transactions on Architecture and Code Optimization10.1145/367800921:3(1-25)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3678009

Index Terms

Effective Lightweight Software Fault Localization based on Test Suite Optimization

Index terms have been assigned to the content through auto-classification.

Recommendations

Impacts of Test Suite's Class Imbalance on Spectrum-Based Fault Localization Techniques
QSIC '13: Proceedings of the 2013 13th International Conference on Quality Software

Spectrum-based fault localization (SBFL) uses the execution results of test cases to debug. There are two types of SBFL techniques: one using conventional slices, and the other using metamorphic slices. This paper investigates the ratio between non-...
Test case selection using multi-criteria optimization for effective fault localization

As spectra-based fault localization techniques report suspicious statements by analyzing the coverage of test cases, the effectiveness of the results is highly dependent on the composition of test suites. This paper proposes an approach for selecting a ...
A test-suite diagnosability metric for spectrum-based fault localization approaches
ICSE '17: Proceedings of the 39th International Conference on Software Engineering

Current metrics for assessing the adequacy of a test-suite plainly focus on the number of components (be it lines, branches, paths) covered by the suite, but do not explicitly check how the tests actually exercise these components and whether they ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ISEC '22: Proceedings of the 15th Innovations in Software Engineering Conference

February 2022

235 pages

ISBN:9781450396189

DOI:10.1145/3511430

Editors:
Saurabh Tiwari
DA-IICT Gandhinagar, India
,
Sanjay Chaudhary
Ahmedabad University, India
,
Chanchal K. Roy
University of Saskatchewan, Canada
,
Meenakshi D'Souza
IIIT Bangalore, India
,
Richa Sharma
Lock Haven University, USA
,
Lov Kumar
BITS-Pilani, Hyderabad, India

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 February 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article
Research
Refereed limited

Conference

ISEC 2022

ISEC 2022: 15th Innovations in Software Engineering Conference

February 24 - 26, 2022

Gandhinagar, India

Acceptance Rates

Overall Acceptance Rate 76 of 315 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
99
Total Downloads

Downloads (Last 12 months)10
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu JZhang ZYang DXu JHe JMao X(2024)Time-Aware Spectrum-Based Bug Localization for Hardware Design Code with Data PurificationACM Transactions on Architecture and Code Optimization10.1145/367800921:3(1-25)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3678009

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents