Bridging the Gap between Different Programming Paradigms in Coverage-based Fault Localization
Authors: 
Feng Li, Meng Wang, Dan HaoAuthors Info & Claims
Feng Li, Meng Wang, Dan HaoAuthors Info & ClaimsPages 75 - 84
Abstract
Fault localization is to identify faulty program elements. Among the large number of fault localization approaches in the literature, coverage-based fault localization, especially spectrum-based fault localization has been intensively studied due to its effectiveness and lightweightness. Despite the rich literature, almost all existing fault localization approaches and studies are conducted on imperative programming languages such as Java and C, leaving a gap in other programming paradigms. In this paper, we aim to study fault localization approaches for the functional programming paradigm, using Haskell language as a representation. We build up the first dataset on real Haskell projects including both real and seeded faults, which enables the research of fault localization for functional languages. With this dataset, we explore fault localization techniques for Haskell. In particular, as typically for SBFL approaches, we study methods for coverage collection as well as formulae for suspiciousness scores computation, and carefully adapt these two components to Haskell considering the language features and characteristics, resulting in a series of adaption approaches and a learning-based approach, which are evaluated on the dataset to demonstrate the promises of the direction.
References
[1]
Rui Abreu, Peter Zoeteweij, Rob Golsteijn, and Arjan JC Van Gemund. 2009. A practical evaluation of spectrum-based fault localization. Journal of Systems and Software 82, 11 (2009), 1780–1792.
[2]
Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2006. An evaluation of similarity coefficients for software fault localization. In 2006 12th Pacific Rim International Symposium on Dependable Computing (PRDC’06). IEEE, 39–46.
[3]
Tien-Duy B. Le, David Lo, Claire Le Goues, and Lars Grunske. 2016. A learning-to-rank based fault localization approach using likely invariants. In Proceedings of the 25th International Symposium on Software Testing and Analysis. 177–188.
[4]
Samuel Benton, Xia Li, Yiling Lou, and Lingming Zhang. 2020. On the Effectiveness of Unified Debugging: An Extensive Study on 16 Program Repair Systems. In 2020 35th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 907–918.
[5]
Hudson Borges and Marco Tulio Valente. 2018. What’s in a GitHub star? understanding repository starring practices in a social coding platform. Journal of Systems and Software 146 (2018), 112–129.
[6]
Rudy Braquehais and Colin Runciman. 2016. FitSpec: refining property sets for functional testing. In Proceedings of the 9th International Symposium on Haskell. 1–12.
[7]
Eric Bruneton, Romain Lenglet, and Thierry Coupaye. 2002. ASM: a code manipulation tool to implement adaptable systems. Adaptable and extensible component systems 30, 19 (2002).
[8]
Bruno Castro, Alexandre Perez, and Rui Abreu. 2019. Pangolin: an SFL-based toolset for feature localization. In 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 1130–1133.
[9]
Yufeng Cheng, Meng Wang, Yingfei Xiong, Dan Hao, and Lu Zhang. 2016. Empirical evaluation of test coverage for functional programs. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST). IEEE, 255–265.
[10]
Koen Claessen and John Hughes. 2000. QuickCheck: a lightweight tool for random testing of Haskell programs. In Proceedings of the fifth ACM SIGPLAN international conference on Functional programming. 268–279.
[11]
Jerome H Friedman. 2002. Stochastic gradient boosting. Computational statistics & data analysis 38, 4 (2002), 367–378.
[12]
Ali Ghanbari, Samuel Benton, and Lingming Zhang. 2019. Practical program repair via bytecode mutation. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 19–30.
[13]
Andy Gill and Colin Runciman. 2007. Haskell program coverage. In Proceedings of the ACM SIGPLAN workshop on Haskell workshop. 1–12.
[14]
Greg4cr. 2022. Defects4J – version 2.0. https://github.com/rjust/defects4j.
[15]
Gustavo Grieco, Martín Ceresa, and Pablo Buiras. 2016. QuickFuzz: An automatic random fuzzer for common file formats. ACM SIGPLAN Notices 51, 12 (2016), 13–20.
[16]
HaFLa. 2022. HaFLa Homepage. https://github.com/Spiridempt/HaFLa.
[17]
Cordelia V Hall, Kevin Hammond, Simon L Peyton Jones, and Philip L Wadler. 1996. Type classes in Haskell. ACM Transactions on Programming Languages and Systems (TOPLAS) 18, 2(1996), 109–138.
[18]
Yue Jia and Mark Harman. 2010. An analysis and survey of the development of mutation testing. IEEE transactions on software engineering 37, 5 (2010), 649–678.
[19]
James A Jones and Mary Jean Harrold. 2005. Empirical evaluation of the tarantula automatic fault-localization technique. In Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering. 273–282.
[20]
Simon Peyton Jones. 2003. Haskell 98 language and libraries: the revised report. Cambridge University Press.
[21]
René Just, Darioush Jalali, and Michael D Ernst. 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 2014 International Symposium on Software Testing and Analysis. 437–440.
[22]
René Just, Darioush Jalali, Laura Inozemtseva, Michael D Ernst, Reid Holmes, and Gordon Fraser. 2014. Are mutants a valid substitute for real faults in software testing?. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 654–665.
[23]
Duc Le, Mohammad Amin Alipour, Rahul Gopinath, and Alex Groce. 2014. Mucheck: An extensible tool for mutation testing of haskell programs. In Proceedings of the 2014 international symposium on software testing and analysis. 429–432.
[24]
Feng Li, Jianyi Zhou, Yinzhu Li, Dan Hao, and Lu Zhang. 2021. AGA: An Accelerated Greedy Additional Algorithm for Test Case Prioritization. IEEE Transactions on Software Engineering(2021).
[25]
Xia Li, Wei Li, Yuqun Zhang, and Lingming Zhang. 2019. Deepfl: Integrating multiple fault diagnosis dimensions for deep fault localization. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis. 169–180.
[26]
Xia Li and Lingming Zhang. 2017. Transforming programs and tests in tandem for fault localization. Proceedings of the ACM on Programming Languages 1, OOPSLA(2017), 1–30.
[27]
Ben Liblit, Mayur Naik, Alice X Zheng, Alex Aiken, and Michael I Jordan. 2005. Scalable statistical bug isolation. Acm Sigplan Notices 40, 6 (2005), 15–26.
[28]
Yiling Lou, Ali Ghanbari, Xia Li, Lingming Zhang, Haotian Zhang, Dan Hao, and Lu Zhang. 2020. Can automated program repair refine fault localization? a unified debugging approach. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. 75–87.
[29]
Agustín Mista, Alejandro Russo, and John Hughes. 2018. Branching processes for quickcheck generators. ACM SIGPLAN Notices 53, 7 (2018), 1–13.
[30]
Seokhyeon Moon, Yunho Kim, Moonzoo Kim, and Shin Yoo. 2014. Ask the mutants: Mutating faulty programs for fault localization. In 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation. IEEE, 153–162.
[31]
Lee Naish, Hua Jie Lee, and Kotagiri Ramamohanarao. 2011. A model for spectra-based software diagnosis. ACM Transactions on software engineering and methodology (TOSEM) 20, 3(2011), 1–32.
[32]
Frolin S Ocariza Jr, Guanpeng Li, Karthik Pattabiraman, and Ali Mesbah. 2016. Automatic fault localization for client-side JavaScript. Software Testing, Verification and Reliability 26, 1(2016), 69–88.
[33]
Mike Papadakis and Yves Le Traon. 2015. Metallaxis-FL: mutation-based fault localization. Software Testing, Verification and Reliability 25, 5-7(2015), 605–628.
[34]
Spencer Pearson, José Campos, René Just, Gordon Fraser, Rui Abreu, Michael D Ernst, Deric Pang, and Benjamin Keller. 2017. Evaluating and improving fault localization. In 2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE). IEEE, 609–620.
[35]
Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, 2011. Scikit-learn: Machine learning in Python. the Journal of machine Learning research 12 (2011), 2825–2830.
[36]
Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A large scale study of programming languages and code quality in github. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering. 155–165.
[37]
Deuslirio Silva-Junior, Plinio S Leitao-Junior, Altino Dantas, Celso G Camilo-Junior, and Rachel Harrison. 2020. Data-flow-based evolutionary fault localization. In Proceedings of the 35th Annual ACM Symposium on Applied Computing. 1963–1970.
[38]
Jeongju Sohn and Shin Yoo. 2017. Fluccs: Using code and change metrics to improve fault localization. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. 273–283.
[39]
Ming Wen, Junjie Chen, Yongqiang Tian, Rongxin Wu, Dan Hao, Shi Han, and Shing-Chi Cheung. 2019. Historical spectrum based fault localization. IEEE Transactions on Software Engineering(2019).
[40]
Ming Wen, Junjie Chen, Rongxin Wu, Dan Hao, and Shing-Chi Cheung. 2018. Context-aware patch generation for better automated program repair. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). IEEE, 1–11.
[41]
Manfred Widera. 2006. Why Testing Matters in Functional Programming. In 7th Symposium on Trends in Functional Programming, University of Nottingham, TFP.
[42]
W Eric Wong, Vidroha Debroy, Ruizhi Gao, and Yihao Li. 2013. The DStar method for effective software fault localization. IEEE Transactions on Reliability 63, 1 (2013), 290–308.
[43]
W Eric Wong, Ruizhi Gao, Yihao Li, Rui Abreu, and Franz Wotawa. 2016. A survey on software fault localization. IEEE Transactions on Software Engineering 42, 8 (2016), 707–740.
[44]
Xiaofeng Xu, Vidroha Debroy, W Eric Wong, and Donghui Guo. 2011. Ties within fault localization rankings: Exposing and addressing the problem. International Journal of Software Engineering and Knowledge Engineering 21, 06(2011), 803–827.
[45]
Yuan Yuan and Wolfgang Banzhaf. 2018. Arja: Automated repair of java programs via multi-objective genetic programming. IEEE Transactions on Software Engineering 46, 10 (2018), 1040–1067.
[46]
Abubakar Zakari, Sai Peck Lee, Rui Abreu, Babiker Hussien Ahmed, and Rasheed Abubakar Rasheed. 2020. Multiple fault localization of software programs: A systematic literature review. Information and Software Technology 124 (2020), 106312.
[47]
Mengshi Zhang, Xia Li, Lingming Zhang, and Sarfraz Khurshid. 2017. Boosting spectrum-based fault localization using pagerank. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. 261–272.
[48]
Xiangyu Zhang, Neelam Gupta, and Rajiv Gupta. 2006. Locating faults through automated predicate switching. In Proceedings of the 28th international conference on Software engineering. 272–281.
[49]
Daming Zou, Jingjing Liang, Yingfei Xiong, Michael D Ernst, and Lu Zhang. 2019. An empirical study of fault localization families and their combinations. IEEE Transactions on Software Engineering(2019).
Index Terms
- Bridging the Gap between Different Programming Paradigms in Coverage-based Fault Localization
Recommendations
Fault density, fault types, and spectra-based fault localization
This paper presents multiple empirical experiments that investigate the impact of fault quantity and fault type on statistical, coverage-based fault localization techniques and fault-localization interference. Fault-localization interference is a ...
On the influence of multiple faults on coverage-based fault localization
ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and AnalysisThis paper presents an empirical study on the effects of the quantity of faults on statistical, coverage-based fault localization techniques. The former belief was that the effectiveness of fault-localization techniques was inversely proportional to the ...
Iterative Feedback-Based Fault Localization Approach
SEAA '11: Proceedings of the 2011 37th EUROMICRO Conference on Software Engineering and Advanced ApplicationsMost current fault-localization approaches merely use the information from testing to locate faults, assuming that test cases satisfying a certain test adequacy criterion can provide adequate information for fault localization. Unfortunately however, ...
Comments
Information & Contributors
Information
Published In
June 2022
291 pages
ISBN:9781450397803
DOI:10.1145/3545258
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 15 September 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
Internetware 2022
Acceptance Rates
Overall Acceptance Rate 55 of 111 submissions, 50%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 69Total Downloads
- Downloads (Last 12 months)15
- Downloads (Last 6 weeks)0
Reflects downloads up to 15 Sep 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format