skip to main content
research-article
Open access

On continuation-passing transformations and expected cost analysis

Published: 19 August 2021 Publication History

Abstract

We define a continuation-passing style (CPS) translation for a typed λ-calculus with probabilistic choice, unbounded recursion, and a tick operator — for modeling cost. The target language is a (non-probabilistic) λ-calculus, enriched with a type of extended positive reals and a fixpoint operator. We then show that applying the CPS transform of an expression M to the continuation λ v. 0 yields the expected cost of M. We also introduce a formal system for higher-order logic, called EHOL, prove it sound, and show it can derive tight upper bounds on the expected cost of classic examples, including Coupon Collector and Random Walk. Moreover, we relate our translation to Kaminski et al.’s ert-calculus, showing that the latter can be recovered by applying our CPS translation to (a generalization of) the classic embedding of imperative programs into λ-calculus. Finally, we prove that the CPS transform of an expression can also be used to compute pre-expectations and to reason about almost sure termination.

Supplementary Material

Auxiliary Presentation Video (icfp21main-p133-p-video.mp4)
Presentation video of our ICFP'21 paper "On Continuation-Passing Transformations and Expected Cost Analysis". We define a continuation-passing style (CPS) translation for a typed λ-calculus with probabilistic choice, unbounded recursion, and a tick operator — for modeling cost. The target language is a (non-probabilistic) λ-calculus, enriched with a type of extended positive reals and a fixpoint operator. We then show that applying the CPS transform of an expression M to the continuation λ v. 0 yields the expected cost of M. We also introduce a formal system for higher-order logic, called EHOL, prove it sound, and show it can derive tight upper bounds on the expected cost of classic examples.
MP4 File (3473592.mp4)
Presentation Videos

References

[1]
S. Agrawal, K. Chatterjee, and P. Novotný. 2018. Lexicographic Ranking Supermartingales: An Efficient Approach to Termination of Probabilistic Programs. PACMPL, 2, POPL (2018), 34:1–34:32. https://doi.org/10.1145/3158122
[2]
A. Aguirre, G. Barthe, M. Gaboardi, D. Garg, and P.-Y. Strub. 2017. A Relational Logic for Higher-order Programs. PACMPL, 1, ICFP (2017), 21:1–21:29. https://doi.org/10.1145/3110265
[3]
Elvira Albert, Puri Arenas, Samir Genaim, German Puebla, and Damiano Zanardini. 2012. Cost analysis of object-oriented bytecode programs. Theor. Comput. Sci., 413, 1 (2012), 142–159. https://doi.org/10.1016/j.tcs.2011.07.009
[4]
Andrew W. Appel. 1991. Compiling with Continuations. Cambridge University Press. https://doi.org/10.1017/CBO9780511609619
[5]
R. Atkey. 2011. Amortised Resource Analysis with Separation Logic. Logical Methods in Computer Science, 7, 2 (2011), https://doi.org/10.2168/LMCS-7(2:17)2011
[6]
M. Avanzini, U. Dal Lago, and A. Ghyselen. 2019. Type-Based Complexity Analysis of Probabilistic Functional Programs. In Proc. of 34 LICS. IEEE, 1–13. https://doi.org/10.1109/LICS.2019.8785725
[7]
M. Avanzini, U. Dal Lago, and G. Moser. 2015. Analysing the Complexity of Functional Programs: Higher-Order Meets First-Order. In Proc. of 20 ICFP. ACM, 152–164. https://doi.org/10.1145/2784731.2784753
[8]
M. Avanzini, U. Dal Lago, and A. Yamada. 2019. On Probabilistic Term Rewriting. SCP, 185 (2019), 102338.
[9]
M. Avanzini, G. Moser, and M. Schaper. 2020. A Modular Cost Analysis for Probabilistic Programs. PACMPL, 4, OOPSLA (2020), 172:1–172:30. https://doi.org/10.1145/3428240
[10]
G. Barthe, B. Grégoire, and S. Z. Béguelin. 2009. Formal Certification of Code-based Cryptographic Proofs. In Proc. of 36 POPL. 90–101. https://doi.org/10.1145/1480881.1480894
[11]
K. Batz, B. L. Kaminski, J.-P. Katoen, and C Matheja. 2021. Relatively Complete Verification of Probabilistic Programs: An Expressive Language for Expectation-based Reasoning. PACMPL, 5, POPL (2021), 1–30. https://doi.org/10.1145/3434320
[12]
O. Bournez and F. Garnier. 2005. Proving Positive Almost-Sure Termination. In Proc. of 16 RTA (LNCS, Vol. 3467). Springer, 323–337. https://doi.org/10.1007/978-3-540-32033-3_24
[13]
T. Brázdil, S. Kiefer, A. Kucera, and I.H. Vareková. 2015. Runtime Analysis of Probabilistic Programs with Unbounded Recursion. J. Comput. Syst. Sci., 81, 1 (2015), 288–310. https://doi.org/10.1016/j.jcss.2014.06.005
[14]
A. Bucciarelli, A. Piperno, and I. Salvo. 2003. Intersection Types and lambda-Definability. MSCS, 13, 1 (2003), 15–53. https://doi.org/10.1017/S0960129502003833
[15]
O. Celiku and A. McIver. 2005. Compositional Specification and Analysis of Cost-Based Properties in Probabilistic Programs. In Proc. International Symposium of Formal Methods Europe (LNCS, Vol. 3582). Springer, 107–122. https://doi.org/10.1007/11526841_9
[16]
A. Chakarov and S. Sankaranarayanan. 2013. Probabilistic Program Analysis with Martingales. In Proc. of 25 CAV (LNCS, Vol. 8044). Springer, 511–526. https://doi.org/10.1007/978-3-642-39799-8_34
[17]
A. Chakarov and S. Sankaranarayanan. 2014. Expectation Invariants for Probabilistic Program Loops as Fixed Points. In Proc. of 21 SAS. 85–100. https://doi.org/10.1007/978-3-319-10936-7_6
[18]
K. Chatterjee, H. Fu, and A. K. Goharshady. 2016. Termination Analysis of Probabilistic Programs Through Positivstellensatz’s. In Proc. of 28 CAV (LNCS, Vol. 9779). Springer, 3–22. https://doi.org/10.1007/978-3-319-41528-4_1
[19]
K. Chatterjee, H. Fu, and A. Murhekar. 2017. Automated Recurrence Analysis for Almost-Linear Expected-Runtime Bounds. In Proc. of 29 CAV (LNCS, Vol. 10426). Springer, 118–139. https://doi.org/10.1007/978-3-319-63387-9_6
[20]
K. Chatterjee, P. Novotný, and D. Zikelic. 2017. Stochastic Invariants for Probabilistic Termination. In Proc. of 44 POPL. ACM, 145–160. https://doi.org/10.1145/3009837.3009873
[21]
P. Cousot and R. Cousot. 1977. Abstract Interpretation: A Unified Lattice Model for Static Analysis of Programs by Construction or Approximation of Fixpoints. In Proc. of 4 POPL. ACM, 238–252. https://doi.org/10.1145/512950.512973
[22]
J. W. Cutler, D. R. Licata, and N. Danner. 2020. Denotational Recurrence Extraction for Amortized Analysis. PACMPL, 4, ICFP (2020), 97:1–97:29. https://doi.org/10.1145/3408979
[23]
U. Dal Lago, C. Faggian, and S. Ronchi Della Rocca. 2021. Intersection Types and (Positive) Almost-sure Termination. PACMPL, 5, POPL (2021), 1–32. https://doi.org/10.1145/3434313
[24]
Ugo Dal Lago and Marco Gaboardi. 2011. Linear Dependent Types and Relative Completeness. Log. Methods Comput. Sci., 8, 4 (2011).
[25]
U. Dal Lago and C. Grellois. 2017. Probabilistic Termination by Monadic Affine Sized Typing. In Proc. of 26 ESOP (LNCS). Springer, 393–419. https://doi.org/10.1145/3293605
[26]
U. Dal Lago and M. Zorzi. 2012. Probabilistic Operational Semantics for the Lambda Calculus. RAIRO - TIA, 46, 3 (2012), 413–450. https://doi.org/10.1051/ita/2012012
[27]
N. A. Danielsson. 2008. Lightweight Semiformal Time Complexity Analysis for Purely Functional Data Structures. In Proc. of 35 POPL. ACM, 133–144. https://doi.org/10.1145/1328438.1328457
[28]
N. Danner, D. R. Licata, and Ramyaa. 2015. Denotational Cost Semantics for Functional Languages with Inductive Types. In Proc. of 20 ICFP. ACM, 140–151. https://doi.org/10.1145/2784731.2784749
[29]
Oliver Danvy and Andrzex Filinski. 1992. Representing Control: a Study of the CPS Transformation. Mathematical Structures in Computer Science, 2, 4 (1992), 361–391. https://doi.org/10.1017/S0960129500001535
[30]
O. Danvy and L. R. Nielsen. 2003. A First-Order One-pass CPS Transformation. TCS, 308, 1-3 (2003), 239–257. https://doi.org/10.1016/S0304-3975(02)00733-8
[31]
K. De Leeuw, E. Moore, C. Shannon, and N. Shapiro. 1956. Computability by Probabilistic Machines. Automata Studies, 34 (1956), 183–198.
[32]
T. Ehrhard, M. Pagani, and C. Tasson. 2014. Probabilistic Coherence Spaces are Fully Abstract for Probabilistic PCF. In POPL. ACM. https://doi.org/10.1145/2535838.2535865
[33]
T. Ehrhard, M. Pagani, and C. Tasson. 2018. Full Abstraction for Probabilistic PCF. J. ACM, 65, 4 (2018), 23:1–23:44. https://doi.org/10.1145/3164540
[34]
J. Esparza, A. Kucera, and R. Mayr. 2005. Quantitative Analysis of Probabilistic Pushdown Automata: Expectations and Variances. In Proc. of 20 LICS. 117–126. https://doi.org/10.1109/LICS.2005.39
[35]
Andrzej Filinski. 1994. Representing Monads. In Conference Record of POPL’94: 21st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, Portland, Oregon, USA, January 17-21, 1994. ACM Press, 446–457.
[36]
T. Gehr, S. Misailovic, and M. Vechev. 2016. PSI: Exact Symbolic Inference for Probabilistic Programs. In Proc. of 28 CAV. 62–83. https://doi.org/10.1007/978-3-319-41528-4_4
[37]
S. Goldwasser and S. Micali. 1984. Probabilistic Encryption. JCSS, 28, 2 (1984), 270–299. https://doi.org/10.1016/0022-0000(84)90070-9
[38]
N. D. Goodman, V. K. Mansinghka, D. M. Roy, K. Bonawitz, and J. B. Tenenbaum. 2008. Church: A Language for Generative Models. In UAI. 220–229.
[39]
J. Goubault-Larrecq. 2019. A Probabilistic and Non-Deterministic Call-by-Push-Value Language. In Proc. of 34 LICS. IEEE, 1–13. https://doi.org/10.1109/LICS.2019.8785809
[40]
C. Heunen, O. Kammar, S. Staton, and H. Yang. 2017. A Convenient Category for Higher-order Probability Theory. In Proc. of 32 LICS. IEEE Computer Society, 1–12. https://doi.org/10.1109/LICS.2017.8005137
[41]
J. Hoffmann, K. Aehlig, and M. Hofmann. 2012. Resource Aware ML. In Proc. of 24 CAV (LNCS, Vol. 7358). Springer, Heidelberg, DE. 781–786. https://doi.org/10.1007/978-3-642-31424-7_64
[42]
J. T. Gill III. 1974. Computational Complexity of Probabilistic Turing Machines. In Proceedings of STOC 1974. ACM, 91–95. https://doi.org/10.1145/800119.803889
[43]
S. Jost, K. Hammond, H.-W. Loidl, and M. Hofmann. 2010. Static Determination of Quantitative Resource Usage for Higher-order Programs. In Proc. of 37 POPL. ACM, 223–236. https://doi.org/10.1145/1706299.1706327
[44]
A. Jung and R. Tix. 1998. The Troublesome Probabilistic Powerdomain. ENTCS, 13 (1998), 70–91. https://doi.org/10.1016/S1571-0661(05)80216-6
[45]
B. L. Kaminski and J.-P. Katoen. 2017. A Weakest Pre-expectation Semantics for Mixed-sign Expectations. In Proc. of 32 LICS. IEEE, 1–12. https://doi.org/10.1109/LICS.2017.8005153
[46]
B. Lucien Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. 2016. Weakest Precondition Reasoning for Expected Run-Times of Probabilistic Programs. In Proc. of 25 ESOP (LNCS, Vol. 9632). Springer, 364–389. https://doi.org/10.1145/3208102
[47]
B. L. Kaminski, J.-P. Katoen, C. Matheja, and F. Olmedo. 2018. Weakest Precondition Reasoning for Expected Runtimes of Randomized Algorithms. JACM, 65, 5 (2018), 30:1–30:68.
[48]
B. L. Kaminski and J.-P. Katoen. 2015. On the Hardness of Almost-Sure Termination. In MFCS 2015, Part I. 307–318. https://doi.org/10.1007/978-3-662-48057-1_24
[49]
J.-P. Katoen. 2016. The Probabilistic Model Checking Landscape. In Proc. of 31 LICS. 31–45. https://doi.org/10.1145/2933575.2934574
[50]
J.-P. Katoen, A. McIver, L. Meinicke, and C.C. Morgan. 2010. Linear-Invariant Generation for Probabilistic Programs: - Automated Support for Proof-Based Methods. In Proc. of 17 SAS. 390–406. https://doi.org/10.1007/978-3-642-15769-1_24
[51]
D. Kozen. 1981. Semantics of Probabilistic Programs. J. Comput. Syst. Sci., 22, 3 (1981), 328–350.
[52]
D. Kozen. 1985. A Probabilistic PDL. JCSC, 30, 2 (1985), 162 – 178. https://doi.org/10.1016/0022-0000(85)90012-1
[53]
A. McIver, C. Morgan, B. L. Kaminski, and J-P Katoen. 2018. A New Proof Rule for Almost-sure Termination. PACMPL, 2, POPL (2018), 33:1–33:28. https://doi.org/10.1145/3158121
[54]
D. Monniaux. 2001. An Abstract Analysis of the Probabilistic Termination of Programs. In Proc. of 8 SAS (LNCS, Vol. 2126). Springer, 111–126. https://doi.org/10.1007/3-540-47764-0_7
[55]
Rajeev Motwani and Prabhakar Raghavan. 1995. Randomized Algorithms. Cambridge University Press.
[56]
P. Narayanan, J. Carette, W. Romano, C. Shan, and R. Zinkov. 2016. Probabilistic Inference by Program Transformation in Hakaru (System Description). In Proc. of 13 FLOPS (LNCS, Vol. 9613). Springer, 62–79. https://doi.org/10.1007/978-3-319-29604-3_5
[57]
N. C. Ngo, Q. Carbonneaux, and J. Hoffmann. 2018. Bounded Expectations: Resource Analysis for Probabilistic Programs. In Proc. of 39 PLDI. ACM, 496–512. https://doi.org/10.1145/3192366.3192394
[58]
Hanne Riis Nielson. 1987. A Hoare-Like Proof System for Analysing the Computation Time of Programs. Sci. Comput. Program., 9, 2 (1987), 107–136. https://doi.org/10.1016/0167-6423(87)90029-3
[59]
F. Olmedo, B. L. Kaminski, J.-P. Katoen, and C. Matheja. 2016. Reasoning About Recursive Probabilistic Programs. In Proc. of 31 LICS. 672–681. https://doi.org/10.1145/2933575.2935317
[60]
G.D. Plotkin. 1975. Call-by-name, Call-by-value and the λ -calculus. TCS, 1, 2 (1975), 125–159. https://doi.org/10.1016/0304-3975(75)90017-1
[61]
M. O. Rabin. 1963. Probabilistic Automata. Information and Control, 6, 3 (1963), 230–245.
[62]
I. Radicek, G. Barthe, M. Gaboardi, D. Garg, and F. Zuleger. 2018. Monadic Refinements for Relational Cost Analysis. PACMPL, 2, POPL (2018), 36:1–36:32. https://doi.org/10.1145/3158124
[63]
N. Ramsey and A. Pfeffer. 2002. Stochastic Lambda Calculus and Monads of Probability Distributions. In Proc. of 29 POPL. ACM, 154–165. https://doi.org/10.1145/503272.503288
[64]
M. Rosendahl. 1989. Automatic Complexity Analysis. In FPCA. 144–156. https://doi.org/10.1145/99370.99381
[65]
N. Saheb-Djahromi. 1978. Probabilistic LCF. In MFCS. 442–451. https://doi.org/10.1007/3-540-08921-7_92
[66]
E. S. Santos. 1969. Probabilistic Turing Machines and Computability. Proc. of AMS, 22, 3 (1969), 704–710.
[67]
A. Ścibior, O. Kammar, M. Vákár, S. Staton, H. Yang, Y. Cai, K. Ostermann, S. K. Moss, C. Heunen, and Z. Ghahramani. 2018. Denotational Validation of Higher-order Bayesian Inference. PACMPL, 2, POPL (2018), 60:1–60:29. https://doi.org/10.1145/3158148
[68]
R. Sedgewick and P. Flajolet. 1996. An Introduction to the Analysis of Algorithms. Addison-Wesley-Longman. isbn:978-0-201-40009-0
[69]
T. Takisaka, Y. Oyabu, N. Urabe, and I. Hasuo. 2018. Ranking and Repulsing Supermartingales for Reachability in Probabilistic Programs. In Proc. of 16 ATVA (LNCS, Vol. 11138). Springer, 476–493. https://doi.org/10.1007/978-3-030-01090-4_28
[70]
J. Tassarotti and R. Harper. 2018. Verified Tail Bounds for Randomized Programs. In Proc. of 9 ITP (LNCS, Vol. 10895). Springer, 560–578. https://doi.org/10.1007/978-3-030-01090-4_28
[71]
D. Tolpin, J.-W. van de Meent, H. Yang, and F. D. Wood. 2016. Design and Implementation of Probabilistic Programming Language Anglican. In Proc. of 28 IFL. ACM, 6:1–6:12. https://doi.org/10.1145/3064899.3064910
[72]
D. Wang, J. Hoffmann, and T. W. Reps. 2018. PMAF: an algebraic framework for static analysis of probabilistic programs. In Proc. of 39 PLDI. 513–528. https://doi.org/10.1145/3192366.3192408
[73]
D. Wang, D. M. Kahn, and J. Hoffmann. 2020. Raising Expectations: Automating Expected Cost Analysis with Types. PACMPL, 4, ICFP (2020), 110:1–110:31. https://doi.org/10.1145/3408992
[74]
P. Wang, H. Fu, A. K. Goharshady, K. Chatterjee, X. Qin, and W. Shi. 2019. Cost Analysis of Nondeterministic Probabilistic Programs. In Proc. of 40 PLDI. ACM, 204–220. https://doi.org/10.1145/3314221.3314581
[75]
B. Wegbreit. 1975. Mechanical Program Analysis. Comm. ACM, 18, 9 (1975), 528–539. https://doi.org/10.1145/361002.361016
[76]
G. Winskel. 1993. The Formal Semantics of Programming Languages. MIT Press.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Programming Languages
Proceedings of the ACM on Programming Languages  Volume 5, Issue ICFP
August 2021
1011 pages
EISSN:2475-1421
DOI:10.1145/3482883
Issue’s Table of Contents
This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 August 2021
Published in PACMPL Volume 5, Issue ICFP

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. CPS transformation
  2. cost analysis
  3. probabilistic programming

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)127
  • Downloads (Last 6 weeks)12
Reflects downloads up to 09 Oct 2024

Other Metrics

Citations

Cited By

View all

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media