article

Speculate: discovering conditional equations and inequalities about black-box functions by reasoning from test results

Authors:

Rudy Braquehais,

Colin RuncimanAuthors Info & Claims

ACM SIGPLAN Notices, Volume 52, Issue 10

Pages 40 - 51

https://doi.org/10.1145/3156695.3122961

Published: 07 September 2017 Publication History

Abstract

This paper presents Speculate, a tool that automatically conjectures laws involving conditional equations and inequalities about Haskell functions. Speculate enumerates expressions involving a given collection of Haskell functions, testing to separate those expressions into apparent equivalence classes. Expressions in the same equivalence class are used to conjecture equations. Representative expressions of different equivalence classes are used to conjecture conditional equations and inequalities. Speculate uses lightweight equational reasoning based on term rewriting to discard redundant laws and to avoid needless testing. Several applications demonstrate the effectiveness of Speculate.

References

[1]

2017. Haskell’s Data.Dynamic library documentation. https://hackage.haskell. org/package/base/docs/Data-Dynamic.html . (2017).

[2]

Franz Baader and Tobias Nipkow. 1999. Term Rewriting and All That. Cambridge University Press.

Digital Library

[3]

Leo Bachmair, Nachum Dershowitz, and David A. Plaisted. 1989. Completion Without Failure. In Resolution Of Equations In Algebraic Structures. Vol. 2. Academic Press, Boston, 1–30.

[4]

Rudy Braquehais and Colin Runciman. 2016. FitSpec: refining property sets for functional testing. In Haskell’16. ACM, 1–12.

Digital Library

[5]

Lukas Bulwahn. 2012. Smart Testing of Functional Programs in Isabelle. In LPAR 2012 (LNCS 7180). Springer, 153–167.

Digital Library

[6]

Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In ICFP’00. ACM, 268–279.

Digital Library

[7]

Koen Claessen, Moa Johansson, Dan Rosén, and Nicholas Smallbone. 2012. HipSpec: Automating inductive proofs of program properties. In Workshop on Automated Theory eXploration: ATX 2012.

[8]

Koen Claessen, Nicholas Smallbone, and John Hughes. 2010. QuickSpec: Guessing Formal Specifications Using Testing. In TAP 2010. Springer, 6–21.

Digital Library

[9]

John Horton Conway. 1971. Regular algebra and finite machines. Chapman and Hall.

[10]

Jonas Duregård, Patrik Jansson, and Meng Wang. 2012. Feat: functional enumeration of algebraic types. In Haskell’12. ACM, 61–72.

Digital Library

[11]

Michael D. Ernst, Jeff H. Perkins, Philip J. Guo, Stephen Mccamant, Carlos Pacheco, Matthew S. Tschantz, and Chen Xiao. 2006. The Daikon system for dynamic detection of likely invariants. Science of Computer Programming 69, 1 (2006), 35–45.

Digital Library

[12]

Moa Johansson, Dan Rosén, Nicholas Smallbone, and Koen Claessen. 2014. Hipster: Integrating Theory Exploration in a Proof Assistant. Springer.

[13]

Donald Knuth and Peter Bendix. 1983. Simple Word Problems in Universal Algebras. In Automation of Reasoning. Springer, 342–376.

[14]

Dexter Kozen. 1994. A completeness theorem for Kleene algebras and the algebra of regular events. Information and Computation 110, 2 (1994), 366–390.

Digital Library

[15]

Fredrik Lindblad. 2007. Property Directed Generation of First-Order Test Data. In TFP’07. 105–123.

[16]

Jason S. Reich, Matthew Naylor, and Colin Runciman. 2013. Advances in Lazy SmallCheck. In IFL’13. Springer, 53–70.

[17]

Colin Runciman, Matthew Naylor, and Fredrik Lindblad. 2008. SmallCheck and Lazy SmallCheck: Automatic Exhaustive Testing for Small Values. In Haskell’08. ACM, 37–48.

Digital Library

[18]

Arto Salomaa. 1966. Two complete axiom systems for the algebra of regular events. Journal of the ACM (JACM) 13, 1 (1966), 158–169.

Digital Library

[19]

Tim Sheard and Simon Peyton Jones. 2002. Template Meta-programming for Haskell. In Haskell’02. ACM, 1–16.

Digital Library

[20]

Nicholas Smallbone. 2011. Property-based testing for functional programs. Licentiate Thesis. Chalmers University of Technology.

[21]

Nicholas Smallbone. 2013. Lightweight verification of functional programs. Ph.D. Dissertation. Chalmers University of Technology.

[22]

Nicholas Smallbone and Moa Johansson. 2017. Quick specifications for the busy programmer. (2017). http://www.cse.chalmers.se/~nicsma/papers/quickspec2.pdf Accepted for publication in JFP, Cambridge University Press.

[23]

The GHC Team. 1992–2017. The Glasgow Haskell Compiler. https://www.haskell. org/ghc/ . (1992–2017).

[24]

Michael Walker and Colin Runciman. 2017. Cheap Remarks about Concurrent Programs. (2017). Accepted for presentation at TFP’17.

Cited By

Johansson MSmallbone N(2021)Conjectures, Tests and Proofs: An Overview of Theory ExplorationElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.341.1341(1-16)Online publication date: 6-Sep-2021
https://doi.org/10.4204/EPTCS.341.1
Gissurarson MRoque DKoppel J(2023)Spectacular: Finding Laws from 25 Trillion Terms2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00035(293-304)Online publication date: Apr-2023
https://doi.org/10.1109/ICST57152.2023.00035
Garrow FLones MStewart RWagner M(2022)Why functional program synthesis matters (in the realm of genetic programming)Proceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3534045(1844-1853)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3534045
Show More Cited By

Index Terms

Speculate: discovering conditional equations and inequalities about black-box functions by reasoning from test results
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Program specifications

Recommendations

Speculate: discovering conditional equations and inequalities about black-box functions by reasoning from test results
Haskell 2017: Proceedings of the 10th ACM SIGPLAN International Symposium on Haskell

This paper presents Speculate, a tool that automatically conjectures laws involving conditional equations and inequalities about Haskell functions. Speculate enumerates expressions involving a given collection of Haskell functions, testing to separate ...
Property-driven testing of black-box functions
FormaliSE '22: Proceedings of the IEEE/ACM 10th International Conference on Formal Methods in Software Engineering

Testing is one of the most frequent means of quality assurance for software. Property-based testing aims at generating test suites for checking code against user-defined properties. Test input generation is, however, most often independent of the ...
FitSpec: refining property sets for functional testing
Haskell '16

This paper presents FitSpec, a tool providing automated assistance in the task of refining sets of test properties for Haskell functions. FitSpec tests mutant variations of functions under test against a given property set, recording any surviving ...

Comments

Information & Contributors

Information

Published In

cover image ACM SIGPLAN Notices

ACM SIGPLAN Notices Volume 52, Issue 10

Haskell '17

October 2017

211 pages

ISSN:0362-1340

EISSN:1558-1160

DOI:10.1145/3156695

Editor:
Andy Gill
University of Kansas, Lawrence, KS

Issue’s Table of Contents

Haskell 2017: Proceedings of the 10th ACM SIGPLAN International Symposium on Haskell
September 2017
211 pages
ISBN:9781450351829
DOI:10.1145/3122955
General Chair:
Iavor Diatchki

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2017

Published in SIGPLAN Volume 52, Issue 10

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
106
Total Downloads

Downloads (Last 12 months)15
Downloads (Last 6 weeks)3

Reflects downloads up to 27 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Johansson MSmallbone N(2021)Conjectures, Tests and Proofs: An Overview of Theory ExplorationElectronic Proceedings in Theoretical Computer Science10.4204/EPTCS.341.1341(1-16)Online publication date: 6-Sep-2021
https://doi.org/10.4204/EPTCS.341.1
Gissurarson MRoque DKoppel J(2023)Spectacular: Finding Laws from 25 Trillion Terms2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00035(293-304)Online publication date: Apr-2023
https://doi.org/10.1109/ICST57152.2023.00035
Garrow FLones MStewart RWagner M(2022)Why functional program synthesis matters (in the realm of genetic programming)Proceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3534045(1844-1853)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3534045
Einarsdóttir SSmallbone NJohansson M(2020)Template-based Theory Exploration: Discovering Properties of Functional Programs by TestingProceedings of the 32nd Symposium on Implementation and Application of Functional Languages10.1145/3462172.3462192(67-78)Online publication date: 2-Sep-2020
https://dl.acm.org/doi/10.1145/3462172.3462192
Walker MRunciman C(2018)Cheap Remarks About Concurrent ProgramsFunctional and Logic Programming10.1007/978-3-319-90686-7_17(264-279)Online publication date: 24-Apr-2018
https://doi.org/10.1007/978-3-319-90686-7_17
Braquehais RRunciman C(2017)ExtrapolateProceedings of the 29th Symposium on the Implementation and Application of Functional Programming Languages10.1145/3205368.3205371(1-11)Online publication date: 30-Aug-2017
https://dl.acm.org/doi/10.1145/3205368.3205371

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents