Context Switch Sensitive Fault Localization
Authors: 
Ferenc Horváth, Roland Aszmann, Péter Attila Soha, Árpád Beszédes, Tibor GyimóthyAuthors Info & Claims
Ferenc Horváth, Roland Aszmann, Péter Attila Soha, Árpád Beszédes, Tibor GyimóthyAuthors Info & ClaimsPages 110 - 119
Abstract
Spectrum-Based Fault Localization (SBFL) is a popular technique to assist developers in pinpointing faulty elements within their code based on test outcomes and code coverage. In this paper, we examine the impact of context switching, i.e., when developers must frequently shift their attention between different code parts (such as methods and classes) while going down the SBFL ranked list to find the faulty statement. The basis of our study is the observation that it requires less effort to investigate statements that are next to each other rather than those in different methods and classes. In particular, we analyse the number of visited methods and classes, as well as the frequency of switches between them during the fault localization process. We found that, in programs from the Defects4J benchmark, developers need to explore 40 methods and 12 classes on average, before finding the faulty statement, leading to 53 method- and 40 class switches, respectively.
We introduce a novel context-aware metric that better approximates the total cost of finding a bug than traditional metrics that solely count the number of statements. Our metric considers both the statement number and the added cost of context switches. We also propose a new strategy for reducing the cost by optimizing the traversal of the elements in the ranked list based on the new context-aware metric. The algorithm not only lowers the number of statements that need to be investigated by 12% but also significantly reduces the number of class and method switches by 52%.
References
[1]
Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan J. C. van Gemund. 2009. A Practical Evaluation of Spectrum-based Fault Localization. J. Syst. Softw. 82, 11 (nov 2009), 1780–1792. https://doi.org/10.1016/j.jss.2009.06.035
[2]
Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2009. Spectrum-based multiple fault localization. In 2009 IEEE/ACM Int. Conf. on Automated Softw. Engineering (CASE). IEEE, 88–99.
[3]
Rui Abreu, Peter Zoeteweij, and Arjan J. C. van Gemund. 2007. On the Accuracy of Spectrum-based Fault Localization. In Proc. of the Testing: Academic and Industrial Conf. Practice and Research Techniques - MUTATION. 89–98.
[4]
Tien-Duy B Le, David Lo, Claire Le Goues, and Lars Grunske. 2016. A learning-to-rank based fault localization approach using likely invariants. In Proc. of the 25th Intl. Sym. on Soft. Testing and Analysis (ISSTA). ACM, 177–188.
[5]
Boris Beizer. 1990. Softw. Test. Techniques. Int. Thomson Computer Press, NY.
[6]
Hong Cheng, David Lo, Yang Zhou, Xiaoyin Wang, and Xifeng Yan. 2009. Identifying Bug Signatures Using Discriminative Graph Mining. In Proc. of the 18th Int. Sym. on Soft. Testing and Analysis(ISSTA ’09). ACM, NY, USA, 141–152.
[7]
William Jay Conover. 1998. Practical nonparametric statistics. Vol. 350. John Wiley & Sons.
[8]
Higor Amario de Souza, Marcos Lordello Chaim, and Fabio Kon. 2016. Spectrum-based Software Fault Localization: A Survey of Techniques, Advances, and Challenges. CoRR abs/1607.04347 (2016). arxiv:1607.04347
[9]
Higor A. de Souza, Danilo Mutti, Marcos L. Chaim, and Fabio Kon. 2018. Contextualizing spectrum-based fault localization. Information and Software Technology 94 (2018), 245–261.
[10]
Carlos Gouveia, Jose Campos, and Rui Abreu. 2013. Using HTML5 visualizations in software fault localization. Proceedings of 2013 1st IEEE Working Conf. on Softw. Visualization, 1–10.
[11]
Robert J Grissom and John J Kim. 2005. Effect sizes for research: A broad practical approach.Lawrence Erlbaum Associates Publishers.
[12]
Brent Hailpern and Peter Santhanam. 2001. Software debugging, testing, and verification. IBM Systems Journal 41, 1 (Dec. 2001), 4–12.
[13]
Mary Jean Harrold, Gregg Rothermel, Rui Wu, and Liu Yi. 1998. An empirical investigation of program spectra. In Proc. of the 1998 ACM SIGPLAN-SIGSOFT workshop PASTE ’98 (Montreal, Quebec, Canada). ACM, 83–90.
[14]
Simon Heiden, Lars Grunske, Timo Kehrer, Fabian Keller, Andre Van Hoorn, Antonio Filieri, and David Lo. 2019. An evaluation of pure spectrum-based fault localization techniques for large-scale software systems. Software: Practice and Experience 49, 8 (2019), 1197–1224.
[15]
Ferenc Horváth, Árpád Beszédes, Béla Vancsics, Gergő Balogh, László Vidács, and Tibor Gyimóthy. 2022. Using Contextual Knowledge in Interactive Fault Localization. Empirical Softw. Engineering 27, 150 (2022), 69.
[16]
René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proc. of the 2014 Int. Sym. on Softw. Testing and Analysis. ACM, 437–440.
[17]
Pavneet S. Kochhar, Xin Xia, David Lo, and Shanping Li. 2016. Practitioners’ expectations on automated fault localization. In Proc. of the 25th Int. Sym. on Softw. Testing and Analysis - ISSTA 2016. ACM, NY, USA, 165–176.
[18]
Priya Parmar and Miral Patel. 2016. Software Fault Localization: A Survey. Int. Journal of Computer Applications 154, 9 (2016), 6–13.
[19]
Chris Parnin and Alessandro Orso. 2011. Are Automated Debugging Techniques Actually Helping Programmers?. In Proc. of the 2011 Int. Sym. on Soft. Testing and Analysis (Toronto, Ontario, Canada). ACM, 199–209.
[20]
Spencer Pearson, José Campos, René Just, Gordon Fraser, Rui Abreu, Michael D. Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and improving fault localization. Proc. of the 39th Int. Conf. on Softw. Eng. (2017), 609–620.
[21]
Alexandre Perez, André Riboira, and Rui Abreu. 2012. A Topology-Based Model for Estimating the Diagnostic Efficiency of Statistics-Based Approaches. In 2012 IEEE 23rd Int. Sym. on Softw. Reliability Engineering Workshops. 171–176.
[22]
Henrique Ribeiro, Roberto Andrioli de Araujo, Marcos Chaim, Higor Souza, and Fabio Kon. 2018. Jaguar: A Spectrum-Based Fault Localization Tool for Real-World Software. In 2018 IEEE 11th Int. Conf. on Softw. Testing, Verification and Validation (ICST). 404–409.
[23]
Higor Souza, Marcelo Lauretto, Fabio Kon, and Marcos Chaim. 2023. Understanding the use of spectrum‐based fault localization. J. of Softw.: Evolution and Process (10 2023), e2622.
[24]
Diomidis Spinellis. 2016. Effective Debugging: 66 Specific Ways to Debug Software and Systems (1st ed.). Addison-Wesley Professional. 256 pages.
[25]
Béla Vancsics, Ferenc Horváth, Attila Szatmári, and Árpád Beszédes. 2022. Fault Localization Using Function Call Frequencies. J. of Sys. and Softw. 193 (2022), 111429.
[26]
Dániel Vince, Attila Szatmári, Ákos Kiss, and Árpád Beszédes. 2022. Division by Zero: Threats and Effects in Spectrum-Based Fault Localization Formulas. In Proc. of the 22nd IEEE Int. Conf. on Softw. Quality, Reliability, and Security (QRS’22) (Guangzhou, China). 221–230.
[27]
W. Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. 2014. The DStar Method for Effective Software Fault Localization. IEEE Trans. Reliability 63 (2014), 290–308.
[28]
W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Trans. Softw. Eng. 42, 8 (2016), 707–740.
[29]
W. Eric Wong, Yu Qi, Lei Zhao, and Kai-Yuan Cai. 2007. Effective Fault Localization using Code Coverage. In 31st Annual Int. Computer Softw. and Applications Conf. (COMPSAC), Vol. 1. 449–456.
[30]
Xin Xia, Lingfeng Bao, David Lo, and Shanping Li. 2016. “Automated Debugging Considered Harmful” Considered Harmful: A User Study Revisiting the Usefulness of Spectra-Based Fault Localization Techniques with Professionals Using Real Bugs from Large Systems. In 2016 IEEE Int. Conf on Softw. Maintenance and Evolution (ICSME). IEEE, 267–278.
[31]
Xiaoyuan Xie, Tsong Y. Chen, Fei-C. Kuo, and Baowen Xu. 2013. A Theoretical Analysis of the Risk Evaluation Formulas for Spectrum-based Fault Localization. ACM Trans. Softw. Eng. Methodol. 22, 4, Article 31 (Oct. 2013), 40 pages.
[32]
Xiaofeng Xu, Vidroha Debroy, W. Eric Wong, and Donghui Guo. 2011. Ties within Fault Localization rankings: Exposing and Addressing the Problem. Int. Journal of Softw. Eng. and Knowledge Eng. 21 (2011), 803–827.
[33]
Jifeng Xuan and Martin Monperrus. 2014. Learning to combine multiple ranking metrics for fault localization. In 2014 IEEE Int. Conf. on Softw. Maintenance and Evolution. IEEE, 191–200.
[34]
Yue Yan, Shujuan Jiang, Yanmei Zhang, Shenggang Zhang, and Cheng Zhang. 2023. A fault localization approach based on fault propagation context. Information and Softw. Technology 160 (2023), 107245.
[35]
Abubakar Zakari, Sai Peck Lee, and Ibrahim A. T. Hashem. 2019. A single fault localization technique based on failed test input. Array 3 (2019), 100008.
[36]
Andreas Zeller. 2009. Why Programs Fail, Second Edition: A Guide to Systematic Debugging (2nd ed.). Morgan Kaufmann Publishers Inc., CA, USA. 424 pages.
[37]
Daming Zou, Jingjing Liang, Yingfei Xiong, Michael D Ernst, and Lu Zhang. 2019. An empirical study of fault localization families and their combinations. IEEE Trans. Softw. Eng. 47, 2 (2019), 332–347.
Index Terms
- Context Switch Sensitive Fault Localization
Recommendations
A fault localization approach based on fault propagation context
Abstract Context:Spectrum-based fault localization (SBFL) performs statistical analysis on the coverage information of failed or passed test cases. It provides the programmer with a guide for fault localization by generating a ...
Fault density, fault types, and spectra-based fault localization
This paper presents multiple empirical experiments that investigate the impact of fault quantity and fault type on statistical, coverage-based fault localization techniques and fault-localization interference. Fault-localization interference is a ...
Using Spectrum-Based Fault Location and Heatmaps to Express Debugging Suggestions to Student Programmers
ACE '17: Proceedings of the Nineteenth Australasian Computing Education ConferenceAutomated grading systems are useful for conveying debugging feedback to students, but the manner in which this feedback is displayed can be problematic for students. Automated systems usually report failures in student code, whether it be the results ...
Comments
Information & Contributors
Information
Published In
June 2024
728 pages
ISBN:9798400717017
DOI:10.1145/3661167
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 18 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
EASE 2024
EASE 2024: 28th International Conference on Evaluation and Assessment in Software Engineering
June 18 - 21, 2024
Salerno, Italy
Acceptance Rates
Overall Acceptance Rate 71 of 232 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 23Total Downloads
- Downloads (Last 12 months)23
- Downloads (Last 6 weeks)2
Reflects downloads up to 06 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format