research-article

MuCheck: an extensible tool for mutation testing of haskell programs

Authors:

Mohammad Amin Alipour,

Rahul Gopinath,

Alex GroceAuthors Info & Claims

ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

Pages 429 - 432

https://doi.org/10.1145/2610384.2628052

Published: 21 July 2014 Publication History

Abstract

This paper presents MuCheck, a mutation testing tool for Haskell programs. MuCheck is a counterpart to the widely used QuickCheck random testing tool for functional programs, and can be used to evaluate the efficacy of QuickCheck property definitions. The tool implements mutation operators that are specifically designed for functional programs, and makes use of the type system of Haskell to achieve a more relevant set of mutants than otherwise possible. Mutation coverage is particularly valuable for functional programs due to highly compact code, referential transparency, and clean semantics; these make augmenting a test suite or specification based on surviving mutants a practical method for improved testing.

References

[1]

A. T. Acree, T. A. Budd, R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Mutation analysis. Technical report, Georgia Institute of Technology, 1979.

[2]

J. H. Andrews, L. C. Briand, and Y. Labiche. Is mutation an appropriate tool for testing experiments? In International Conference on Software Engineering, pages 402–411, 2005.

Digital Library

[3]

N. Broberg. The haskell-src-exts package. http: //hackage.haskell.org/package/haskell-src-exts.

[4]

K. Claessen and J. Hughes. Quickcheck: a lightweight tool for random testing of haskell programs. SIGPLAN Not., pages 53–64, 2011.

Digital Library

[5]

R. DeMillo, R. Lipton, and F. Sayward. Hints on test data selection: Help for the practicing programmer. Computer, 11(4):34–41, 1978.

Digital Library

[6]

R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on test data selection: Help for the practicing programmer. Computer, 11(4):34–41, 1978.

Digital Library

[7]

M. Gligoric, A. Groce, C. Zhang, R. Sharma, M. A. Alipour, and D. Marinov. Comparing non-adequate test suites using coverage criteria. In ACM International Symposium on Software Testing and Analysis. ACM, 2013.

Digital Library

[8]

R. Gopinath, M. A. Alipour, I. Ahmed, C. Jensen, and A. Groce. An empirical comparison of mutant selection approaches. under submission.

[9]

R. Gopinath, C. Jensen, and A. Groce. Mutant census: An empirical examination of the competent programmer hypothesis. In Technical Report, School of EECS, Oregon State University.

[10]

D. Gorin. The hint package. http://hackage.haskell.org/package/hint.

[11]

A. Groce, C. Zhang, E. Eide, Y. Chen, and J. Regehr. Swarm testing. In International Symposium on Software Testing and Analysis, pages 78–88, 2012.

Digital Library

[12]

R. Hamlet. Testing programs with the aid of a compiler. Software Engineering, IEEE Transactions on, SE-3(4):279–290, 1977.

Digital Library

[13]

Y. Jia and M. Harman. An analysis and survey of the development of mutation testing. IEEE TSE, 37(5):649–678, 2011.

Digital Library

[14]

R. Lämmel and S. P. Jones. Scrap your boilerplate: a practical design pattern for generic programming. In Proceedings of the 2003 ACM SIGPLAN international workshop on Types in languages design and implementation, TLDI ’03, pages 26–37, 2003.

Digital Library

[15]

A. Mathur. Performance, effectiveness, and reliability issues in software testing. In Computer Software and Applications Conference, 1991. COMPSAC ’91., Proceedings of the Fifteenth Annual International, pages 604–605, 1991.

[16]

A. Offutt, G. Rothermel, and C. Zapf. An experimental evaluation of selective mutation. In Software Engineering, 1993. Proceedings., 15th International Conference on, pages 100–107, 1993.

Digital Library

[17]

W. Wong and A. P. Mathur. Reducing the cost of mutation testing: An empirical study. Journal of Systems and Software, 31(3):185 – 196, 1995.

Digital Library

[18]

L. Zhang, M. Gligoric, D. Marinov, and S. Khurshid. Operator-based and random mutant selection: Better together. In IEEE/ACM International Conference on Automated Software Engineering. ACM, 2013.

Digital Library

Cited By

Deb SJain Kvan Tonder RLe Goues CGroce A(2024)Syntax Is All You Need: A Universal-Language Approach to Mutant GenerationProceedings of the ACM on Software Engineering10.1145/36437561:FSE(654-674)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643756
Jovanovic ASullivan A(2024)Mutation testing for temporal alloy models (extended version)Software and Systems Modeling10.1007/s10270-024-01220-xOnline publication date: 28-Oct-2024
https://doi.org/10.1007/s10270-024-01220-x
Moradi Moghadam MBagherzadeh MKhatchadourian RBagheri HChandra SBlincoe KTonella P(2023)𝜇Akka: Mutation Testing for Actor Concurrency in Akka using Real-World BugsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616362(262-274)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616362
Show More Cited By

Index Terms

MuCheck: an extensible tool for mutation testing of haskell programs
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. General programming languages
      1. Language types

Recommendations

Faster mutation testing inspired by test prioritization and reduction
ISSTA 2013: Proceedings of the 2013 International Symposium on Software Testing and Analysis

Mutation testing is a well-known but costly approach for determining test adequacy. The central idea behind the approach is to generate mutants, which are small syntactic transformations of the program under test, and then to measure for a given test ...
High coverage testing of Haskell programs
ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and Analysis

This paper presents a new lightweight technique for automatically generating high coverage test suites for Haskell library code. Our approach combines four main features to increase test coverage: (1) automatically inferring the constructors and ...
Mutation Testing for Aspect-Oriented Programs
ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation

Mutation testing has been shown to be one of the strongest testing criteria for the evaluation of both programs and test suites. Comprehensive sets of mutants require strong test sets to achieve acceptable testing coverage. Moreover, mutation operators ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISSTA 2014: Proceedings of the 2014 International Symposium on Software Testing and Analysis

July 2014

460 pages

ISBN:9781450326452

DOI:10.1145/2610384

General Chair:
Corina S. Păsăreanu
NASA Ames, USA
,
Program Chair:
Darko Marinov
University of Illinois at Urbana-Champaign, USA

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

In-Cooperation

SIGPLAN: ACM Special Interest Group on Programming Languages

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ISSTA '14

Sponsor:

SIGSOFT

ISSTA '14: International Symposium on Software Testing and Analysis

July 21 - 25, 2014

CA, San Jose, USA

Acceptance Rates

Overall Acceptance Rate 58 of 213 submissions, 27%

Upcoming Conference

ISSTA '25

Sponsor:
sigsoft

34th ACM SIGSOFT International Symposium on Software Testing and Analysis

June 25 - 28, 2025

Trondheim , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
249
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)1

Reflects downloads up to 16 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Deb SJain Kvan Tonder RLe Goues CGroce A(2024)Syntax Is All You Need: A Universal-Language Approach to Mutant GenerationProceedings of the ACM on Software Engineering10.1145/36437561:FSE(654-674)Online publication date: 12-Jul-2024
https://dl.acm.org/doi/10.1145/3643756
Jovanovic ASullivan A(2024)Mutation testing for temporal alloy models (extended version)Software and Systems Modeling10.1007/s10270-024-01220-xOnline publication date: 28-Oct-2024
https://doi.org/10.1007/s10270-024-01220-x
Moradi Moghadam MBagherzadeh MKhatchadourian RBagheri HChandra SBlincoe KTonella P(2023)𝜇Akka: Mutation Testing for Actor Concurrency in Akka using Real-World BugsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616362(262-274)Online publication date: 30-Nov-2023
https://dl.acm.org/doi/10.1145/3611643.3616362
Jovanovic ASullivan A(2023)Mutation Testing for Temporal Alloy Models2023 ACM/IEEE 26th International Conference on Model Driven Engineering Languages and Systems (MODELS)10.1109/MODELS58315.2023.00013(228-238)Online publication date: 1-Oct-2023
https://doi.org/10.1109/MODELS58315.2023.00013
Zhuang BPerretta JGuha ABell J(2023)A Tool for Mutation Analysis in Racket2023 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW)10.1109/ICSTW58534.2023.00061(308-313)Online publication date: Apr-2023
https://doi.org/10.1109/ICSTW58534.2023.00061
Li FWang MHao D(2022)Bridging the Gap between Different Programming Paradigms in Coverage-based Fault LocalizationProceedings of the 13th Asia-Pacific Symposium on Internetware10.1145/3545258.3545272(75-84)Online publication date: 11-Jun-2022
https://dl.acm.org/doi/10.1145/3545258.3545272
Groce AAhmed IFeist JGrieco GGesi JMeidani MChen Q(2021)Evaluating and Improving Static Analysis Tools Via Differential Mutation Analysis2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS)10.1109/QRS54544.2021.00032(207-218)Online publication date: Dec-2021
https://doi.org/10.1109/QRS54544.2021.00032
Wong CMeinicke JChen LDiniz JKästner CFigueiredo EDevanbu PCohen MZimmermann T(2020)Efficiently finding higher-order mutantsProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3368089.3409713(1165-1177)Online publication date: 8-Nov-2020
https://dl.acm.org/doi/10.1145/3368089.3409713
Gómez-Abajo PGuerra ELara JMerayo M(2020)Wodel-Test: a model-based framework for language-independent mutation testingSoftware and Systems Modeling10.1007/s10270-020-00827-0Online publication date: 17-Oct-2020
https://doi.org/10.1007/s10270-020-00827-0
Lazarek LKing ASundar SFindler RDimoulas C(2019)Does blame shifting work?Proceedings of the ACM on Programming Languages10.1145/33711334:POPL(1-29)Online publication date: 20-Dec-2019
https://dl.acm.org/doi/10.1145/3371133
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents