skip to main content
10.1145/3192366.3192394acmconferencesArticle/Chapter ViewAbstractPublication PagespldiConference Proceedingsconference-collections
research-article
Public Access

Bounded expectations: resource analysis for probabilistic programs

Published: 11 June 2018 Publication History

Abstract

This paper presents a new static analysis for deriving upper bounds on the expected resource consumption of probabilistic programs. The analysis is fully automatic and derives symbolic bounds that are multivariate polynomials in the inputs. The new technique combines manual state-of-the-art reasoning techniques for probabilistic programs with an effective method for automatic resource-bound analysis of deterministic programs. It can be seen as both, an extension of automatic amortized resource analysis (AARA) to probabilistic programs and an automation of manual reasoning for probabilistic programs that is based on weakest preconditions. An advantage of the technique is that it combines the clarity and compositionality of a weakest-precondition calculus with the efficient automation of AARA. As a result, bound inference can be reduced to off-the-shelf LP solving in many cases and automatically-derived bounds can be interactively extended with standard program logics if the automation fails. Building on existing work, the soundness of the analysis is proved with respect to an operational semantics that is based on Markov decision processes. The effectiveness of the technique is demonstrated with a prototype implementation that is used to automatically analyze 39 challenging probabilistic programs and randomized algorithms. Experiments indicate that the derived constant factors in the bounds are very precise and even optimal for some programs.

Supplementary Material

WEBM File (p496-ngo.webm)

References

[1]
Elvira Albert, Jesús Correas Fernández, and Guillermo Román-Díez. 2015. Non-cumulative Resource Analysis. In Tools and Algorithms for the Construction and Analysis of Systems - 21st International Conference, (TACAS’15).
[2]
Diego Esteban Alonso-Blas and Samir Genaim. 2012. On the Limits of the Classical Approach to Cost Analysis. In 19th Int. Static Analysis Symp. (SAS’12).
[3]
R. B. Ash and C. Doléans-Dade. 2000. Probability and Measure Theory. Academic Press.
[4]
Robert Atkey. 2010. Amortised Resource Analysis with Separation Logic. In 19th Euro. Symp. on Prog. (ESOP’10).
[5]
Martin Avanzini, Ugo Dal Lago, and Georg Moser. 2012. Analysing the Complexity of Functional Programs: Higher-Order Meets First-Order. In 29th Int. Conf. on Functional Programming (ICFP’15).
[6]
Martin Avanzini and Georg Moser. 2013. A Combination Framework for Complexity. In 24th International Conference on Rewriting Techniques and Applications (RTA’13).
[7]
Gilles Barthe, Thomas Espitau, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016. Formal Certification of Randomized Algorithms. Technical Report. http://justinh.su/files/papers/ellora.pdf.
[8]
Gilles Barthe, Marco Gaboardi, Emilio Jesús Gallego Arias, Justin Hsu, César Kunz, and Pierre-Yves Strub. 2014. Proving Differential Privacy in Hoare Logic. In Proceedings of the 2014 IEEE 27th Computer Security Foundations Symposium (CSF ’14). IEEE Computer Society.
[9]
Gilles Barthe, Benjamin Grégoire, and Santiago Zanella Béguelin. 2009. Formal Certification of Code-based Cryptographic Proofs. In Proceedings of the 36th Annual Symposium on Principles of Programming Languages (POPL’09). ACM, New York, NY, USA.
[10]
K. Batz, B. L. Kaminski, J.-P. Katoen, and C. Matheja. 2018. How long, O Bayesian network, will I sample thee? A program analysis perspective on expected sampling times. ArXiv e-prints (Feb. 2018). arXiv: cs.PL/1802.10433 To appear at ESOP’18.
[11]
Ralph Benzinger. 2004. Automated Higher-Order Complexity Analysis. Theor. Comput. Sci. 318, 1-2 (2004).
[12]
Régis Blanc, Thomas A. Henzinger, Thibaud Hottelier, and Laura Kovács. 2010. ABC: Algebraic Bound Computation for Loops. In Logic for Prog., AI., and Reasoning - 16th Int. Conf. (LPAR’10).
[13]
Mateus Borges, Antonio Filieri, Marcelo d’Amorim, Corina S. Pasareanu, and Willem Visser. 2014. Compositional solution space quantification for probabilistic software analysis. In Conference on Programming Language Design and Implementation (PLDI’14).
[14]
Tomás Brázdil, Stefan Kiefer, Antonín Kucera, and Ivana Hutarová Vareková. 2015. Runtime analysis of probabilistic programs with unbounded recursion. J. Comput. Syst. Sci. 81, 1 (2015).
[15]
Marc Brockschmidt, Fabian Emmes, Stephan Falke, Carsten Fuhs, and Jürgen Giesl. 2014. Alternating Runtime and Size Complexity Analysis of Integer Programs. In 20th Int. Conf. on Tools and Alg. for the Constr. and Anal. of Systems (TACAS’14).
[16]
Quentin Carbonneaux, Jan Hoffmann, Tahina Ramananandro, and Zhong Shao. 2014. End-to-End Verification of Stack-Space Bounds for C Programs. In 35th Conference on Programming Language Design and Implementation (PLDI’14). Artifact submitted and approved.
[17]
Quentin Carbonneaux, Jan Hoffmann, Thomas Reps, and Zhong Shao. 2017. Automated Resource Analysis with Coq Proof Objects. In 29th International Conference on Computer-Aided Verification (CAV’17).
[18]
Quentin Carbonneaux, Jan Hoffmann, and Zhong Shao. 2015. Compositional Certified Resource Bounds. In 36th Conference on Programming Language Design and Implementation (PLDI’15). Artifact submitted and approved.
[19]
Orieta Celiku and Annabelle McIver. 2005. Compositional Specification and Analysis of Cost-Based Properties in Probabilistic Programs. In Formal Methods, International Symposium of Formal Methods Europe (FM’05).
[20]
Pavol Cerný, Thomas A. Henzinger, Laura Kovács, Arjun Radhakrishna, and Jakob Zwirchmayr. 2015. Segment Abstraction for WorstCase Execution Time Analysis. In 24th European Symposium on Programming (ESOP’15).
[21]
Aleksandar Chakarov and Sriram Sankaranarayanan. 2013. Probabilistic Program Analysis using Martingales. In Computer-Aided Verification (CAV’13) (Lecture Notes in Computer Science), Vol. 8044. SpringerVerlag.
[22]
Aleksandar Chakarov and Sriram Sankaranarayanan. 2014. Expectation Invariants as Fixed Points of Probabilistic Programs. In Static Analysis Symposium (SAS’14) (Lecture Notes in Computer Science), Vol. 8723. Springer-Verlag.
[23]
Arthur Charguéraud and François Pottier. 2015. Machine-Checked Verification of the Correctness and Amortized Complexity of an Efficient Union-Find Implementation. In Interactive Theorem Proving -6th International Conference (ITP’15).
[24]
Krishnendu Chatterjee, Hongfei Fu, and Amir Kafshdar Goharshady. 2016. Termination Analysis of Probabilistic Programs Through Positivstellensatz’s. In Computer Aided Verification - 28th International Conference (CAV’16).
[25]
Krishnendu Chatterjee, Hongfei Fu, and Amir Kafshdar Goharshady. 2017. Non-polynomial Worst-Case Analysis of Recursive Programs. In Computer Aided Verification - 29th Int. Conf. (CAV’17).
[26]
Krishnendu Chatterjee, Hongfei Fu, and Aniket Murhekar. 2017. Automated Recurrence Analysis for Almost-Linear Expected-Runtime Bounds. In Computer Aided Verification - 29th Int. Conf. (CAV’17).
[27]
Ezgi Çiçek, Deepak Garg, and Umut A. Acar. 2015. Refinement Types for Incremental Computational Complexity. In 24th European Symposium on Programming (ESOP’15).
[28]
Patrick Cousot and Michael Monerau. 2012. Probabilistic Abstract Interpretation. In Programming Languages and Systems - 21st European Symposium on Programming (ESOP’12).
[29]
Karl Crary and Stephanie Weirich. 2000. Resource Bound Certification. In 27th ACM Symp. on Principles of Prog. Langs. (POPL’00).
[30]
Nils Anders Danielsson. 2008. Lightweight Semiformal Time Complexity Analysis for Purely Functional Data Structures. In 35th ACM Symp. on Principles Prog. Langs. (POPL’08).
[31]
Norman Danner, Daniel R. Licata, and Ramyaa Ramyaa. 2012. Denotational Cost Semantics for Functional Languages with Inductive Types. In 29th Int. Conf. on Functional Programming (ICFP’15).
[32]
D. Dubhashi and A. Panconesi. 2009. Concentration of Measure for the Analysis of Randomized Algorithms. Cambridge University Press (2009).
[33]
Javier Esparza, Antonín Kucera, and Richard Mayr. 2005. Quantitative Analysis of Probabilistic Pushdown Automata: Expectations and Variances. In 20th IEEE Symposium on Logic in Computer Science (LICS’05).
[34]
Luis María Ferrer Fioriti and Holger Hermanns. 2015. Probabilistic Termination: Soundness, Completeness, and Compositionality. In Proceedings of the 42Nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’15). ACM, New York, NY, USA, 13.
[35]
Philippe Flajolet, Bruno Salvy, and Paul Zimmermann. 1991. Automatic Average-case Analysis of Algorithms. Theoret. Comput. Sci. 79, 1 (1991).
[36]
Antonio Flores-Montoya and Reiner Hähnle. 2014. Resource Analysis of Complex Programs with Cost Equations. In Programming Languages and Systems - 12th Asian Symposiu (APLAS’14).
[37]
Florian Frohn, M. Naaf, Jera Hensel, Marc Brockschmidt, and Jürgen Giesl. 2016. Lower Runtime Bounds for Integer Programs. In Automated Reasoning - 8th International Joint Conference (IJCAR’16).
[38]
Timon Gehr, Sasa Misailovic, and Martin T. Vechev. 2016. PSI: Exact Symbolic Inference for Probabilistic Programs. In Computer Aided Verification - 28th International Conference, CAV 2016, Toronto, ON, Canada, July 17-23, 2016, Proceedings, Part I.
[39]
Jaco Geldenhuys, Matthew B. Dwyer, and Willem Visser. 2012. Probabilistic symbolic execution. In International Symposium on Software Testing and Analysis (ISSTA’12).
[40]
Zoubin Ghahramani. 2015. Probabilistic machine learning and artificial intelligence. Nature 521 (2015).
[41]
Andrew D. Gordon, Thomas A. Henzinger, Aditya V. Nori, and Sriram K. Rajamani. 2014. Probabilistic Programming. In Proceedings of the on Future of Software Engineering (FOSE’14).
[42]
F. Gretz, J. Katoen, and A. McIver. 2014. Operational versus Weakest Pre–Expectation Semantics for the Probabilistic Guarded Command Language. Performance Evaluation 73 (2014).
[43]
G. Grimmett and D. Stirzaker. 1992. Probability and Random Processes. Oxford University Press.
[44]
Bernd Grobauer. 2001. Cost Recurrences for DML Programs. In 6th Int. Conf. on Funct. Prog. (ICFP’01).
[45]
Sumit Gulwani, Krishna K. Mehra, and Trishul M. Chilimbi. 2009. SPEED: Precise and Efficient Static Estimation of Program Computational Complexity. In 36th ACM Symp. on Principles of Prog. Langs. (POPL’09).
[46]
Andreas Haeberlen, Benjamin C. Pierce, and Arjun Narayan. 2011. Differential Privacy Under Fire. In Proceedings of the 20th USENIX Conference on Security (SEC’11). USENIX Association, 1.
[47]
Jan Hoffmann, Klaus Aehlig, and Martin Hofmann. 2011. Multivariate Amortized Resource Analysis. In 38th Symposium on Principles of Programming Languages (POPL’11).
[48]
Jan Hoffmann, Klaus Aehlig, and Martin Hofmann. 2012. Multivariate Amortized Resource Analysis. ACM Trans. Program. Lang. Syst. (2012).
[49]
Jan Hoffmann, Ankush Das, and Shu-Chun Weng. 2017. Towards Automatic Resource Bound Analysis for OCaml. In 44th Symposium on Principles of Programming Languages (POPL’17).
[50]
Jan Hoffmann and Martin Hofmann. 2010. Amortized Resource Analysis with Polynomial Potential. In 19th European Symposium on Programming (ESOP’10).
[51]
Jan Hoffmann and Zhong Shao. 2014. Type-Based Amortized Resource Analysis with Integers and Arrays. In 12th International Symposium on Functional and Logic Programming (FLOPS’14).
[52]
Jan Hoffmann and Zhong Shao. 2015. Type-Based Amortized Resource Analysis with Integers and Arrays. J. Funct. Program. (2015).
[53]
Martin Hofmann and Steffen Jost. 2003. Static Prediction of Heap Space Usage for First-Order Functional Programs. In 30th ACM Symp. on Principles of Prog. Langs. (POPL’03).
[54]
Martin Hofmann and Steffen Jost. 2006. Type-Based Amortised HeapSpace Analysis. In 15th Euro. Symp. on Prog. (ESOP’06).
[55]
Martin Hofmann and Georg Moser. 2014. Amortised Resource Analysis and Typed Polynomial Interpretations. In Rewriting and Typed Lambda Calculi (RTA-TLCA;14).
[56]
Martin Hofmann and Georg Moser. 2015. Multivariate Amortised Resource Analysis for Term Rewrite Systems. In 13th International Conference on Typed Lambda Calculi and Applications (TLCA’15).
[57]
Martin Hofmann and Dulma Rodriguez. 2013. Automatic Type Inference for Amortised Heap-Space Analysis. In 22nd Euro. Symp. on Prog. (ESOP’13).
[58]
B. Jeannet and A. Miné. 2009. APRON: A library of numerical abstract domains for static analysis. In Proceedings Computer Aided Verification CAV’2009. LNCS.
[59]
Steffen Jost, Kevin Hammond, Hans-Wolfgang Loidl, and Martin Hofmann. 2010. Static Determination of Quantitative Resource Usage for Higher-Order Programs. In 37th ACM Symp. on Principles of Prog. Langs. (POPL’10).
[60]
Steffen Jost, Hans-Wolfgang Loidl, Kevin Hammond, Norman Scaife, and Martin Hofmann. 2009. Carbon Credits for Resource-Bounded Computations using Amortised Analysis. In 16th Symp. on Form. Meth. (FM’09).
[61]
Benjamin Lucien Kaminski, Joost-Pieter Katoen, Christoph Matheja, and Federico Olmedo. 2016. Weakest Precondition Reasoning for Expected Run–Times of Probabilistic Programs. In Proceedings of the European Symposium on Programming Languages and Systems (ESOP’16). Springer.
[62]
Joost-Pieter Katoen. 2016. The Probabilistic Model Checking Landscape. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, LICS ’16, New York, NY, USA, July 5-8, 2016.
[63]
Joost-Pieter Katoen, Annabelle McIver, Larissa Meinicke, and Carroll C. Morgan. 2010. Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods. In Static Analysis - 17th International Symposium (SAS’10).
[64]
Zachary Kincaid, Jason Breck, Ashkan Forouhi Boroujeni, and Thomas Reps. 2017. Compositional Recurrence Analysis Revisited. In Conference on Programming Language Design and Implementation (PLDI’17).
[65]
Dexter Kozen. 1981. Semantics of Probabilistic Programs. J. Comput. Syst. Sci. 22, 3 (1981).
[66]
Ugo Dal Lago and Marco Gaboardi. 2011. Linear Dependent Types and Relative Completeness. In 26th IEEE Symp. on Logic in Computer Science (LICS’11).
[67]
Ugo Dal Lago and Barbara Petit. 2013. The Geometry of Types. In 40th ACM Symp. on Principles Prog. Langs. (POPL’13).
[68]
Benjamin Lichtman and Jan Hoffmann. 2017. Arrays and References in Resource Aware ML. In 2nd International Conference on Formal Structures for Computation and Deduction (FSCD’17).
[69]
Ravichandhran Madhavan, Sumith Kulal, and Viktor Kuncak. 2017. Contract-based resource verification for higher-order functions with memoization. In Proceedings of the 44th Symposium on Principles of Programming Languages (POPL’17).
[70]
Annabelle McIver and Carroll Morgan. 2004. Abstraction, Refinement and Proof For Probabilistic Systems (Monographs in Computer Science). Springer Verlag.
[71]
David Monniaux. 2001. Backwards Abstract Interpretation of Probabilistic Programs. In Programming Languages and Systems, 10th European Symposium on Programming (ESOP’01).
[72]
David Monniaux. 2005. Abstract interpretation of programs as Markov decision processes. Sci. Comput. Program. 58, 1-2 (2005).
[73]
Van Chan Ngo, Quentin Carbonneaux, and Jan Hoffmann. 2017. Bounded Expectations: Resource Analysis for Probabilistic Programs. CoRR abs/1711.08847 (2017). arXiv: 1711.08847 http://arxiv.org/abs/ 1711.08847
[74]
Van Chan Ngo, Mario Dehesa-Azuara, Matthew Fredrikson, and Jan Hoffmann. 2017. Verifying and Synthesizing Constant-Resource Implementations with Types. In 38th IEEE Symposium on Security and Privacy (S&P ’17).
[75]
Van Chan Ngo and Axel Legay. 2018. PSCV: A Runtime Verification Tool for Probabilistic SystemC Models. J. of Software: Evolution and Process (2018).
[76]
Van Chan Ngo, Axel Legay, and Vania Joloboff. 2016. PSCV: A Runtime Verification Tool for Probabilistic SystemC Models. In 28th International Conference on Computer Aided Verification (CAV’16).
[77]
Van Chan Ngo, Axel Legay, and Jean Quilbeuf. 2016. Statistical Model Checking for SystemC Models. In 17th IEEE High Assurance Systems Engineering Symposium (HASE’16).
[78]
Hanne Riis Nielson. 1987. A Hoare-Like Proof System for Analysing the Computation Time of Programs. Sci. Comput. Program. 9, 2 (1987).
[79]
Tobias Nipkow. 2015. Amortized Complexity Verified. In Interactive Theorem Proving - 6th International Conference (ITP’15).
[80]
Lars Noschinski, Fabian Emmes, and Jürgen Giesl. 2013. Analyzing Innermost Runtime Complexity of Term Rewriting by Dependency Pairs. J. Autom. Reasoning 51, 1 (2013).
[81]
Federico Olmedo, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. 2016. Reasoning about Recursive Probabilistic Programs. In Proceedings of the 31st Annual ACM/IEEE Symposium on Logic in Computer Science, LICS ’16, New York, NY, USA, July 5-8, 2016.
[82]
Avi Pfeffer. 2016. Practical Probabilistic Programming. Manning. https: //books.google.com/books?id=qyfksgEACAAJ
[83]
Sriram Sankaranarayanan, Aleksandar Chakarov, and Sumit Gulwani. 2013. Static Analysis for Probabilistic Programs: Inferring Whole Program Properties from Finitely Many Paths. In ACM conference on Programming Languages Design and Implementation (PLDI’13). ACM Press.
[84]
Hugo R. Simões, Pedro B. Vasconcelos, Mário Florido, Steffen Jost, and Kevin Hammond. 2012. Automatic Amortised Analysis of Dynamic Memory Allocation for Lazy Functional Programs. In 17th Int. Conf. on Funct. Prog. (ICFP’12).
[85]
Moritz Sinn, Florian Zuleger, and Helmut Veith. 2014. A Simple and Scalable Approach to Bound Analysis and Amortized Complexity Analysis. In Computer Aided Verification - 26th Int. Conf. (CAV’14).
[86]
The CLP Team. 2018. CLP. https://projects.coin-or.org/Clp . (2018).
[87]
Pedro Vasconcelos. 2008. Space Cost Analysis Using Sized Types. Ph.D. Dissertation. School of Computer Science, University of St Andrews.
[88]
Pedro B. Vasconcelos, Steffen Jost, Mário Florido, and Kevin Hammond. 2015. Type-Based Allocation Analysis for Co-recursion in Lazy Functional Languages. In 24th European Symposium on Programming (ESOP’15).
[89]
Florian Zuleger, Moritz Sinn, Sumit Gulwani, and Helmut Veith. 2011. Bound Analysis of Imperative Programs with the Size-change Abstraction. In 18th Int. Static Analysis Symp. (SAS’11).
[90]
Ezgi Çiçek, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Jan Hoffmann. 2017. Relational Cost Analysis. In 44th Symposium on Principles of Programming Languages (POPL’17).

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
PLDI 2018: Proceedings of the 39th ACM SIGPLAN Conference on Programming Language Design and Implementation
June 2018
825 pages
ISBN:9781450356985
DOI:10.1145/3192366
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 June 2018

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. Probabilistic programming
  2. Resource bound analysis
  3. Static analysis

Qualifiers

  • Research-article

Funding Sources

Conference

PLDI '18
Sponsor:

Acceptance Rates

Overall Acceptance Rate 406 of 2,067 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)147
  • Downloads (Last 6 weeks)27
Reflects downloads up to 14 Sep 2024

Other Metrics

Citations

Cited By

View all

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media