skip to main content
research-article
Open access

Higher-order probabilistic adversarial computations: categorical semantics and program logics

Published: 19 August 2021 Publication History

Abstract

Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. These computations arise routinely in several domains, including security, privacy and machine learning.
In this paper, we develop program logics for reasoning about adversarial computations in a higher-order setting. Our logics are built on top of a simply typed λ-calculus extended with a graded monad for probabilities and state. The grading is used to model and restrict the memory footprint and the cost (in terms of oracle calls) of computations. Under this view, an adversary is a higher-order expression that expects as arguments the code of its oracles. We develop unary program logics for reasoning about error probabilities and expected values, and a relational logic for reasoning about coupling-based properties. All logics feature rules for adversarial computations, and yield guarantees that are valid for all adversaries that satisfy a fixed resource policy. We prove the soundness of the logics in the category of quasi-Borel spaces, using a general notion of graded predicate liftings, and we use logical relations over graded predicate liftings to establish the soundness of proof rules for adversaries. We illustrate the working of our logics with simple but illustrative examples.

Supplementary Material

Auxiliary Presentation Video (icfp21main-p187-p-video.mp4)
This is a presentation of our paper at ICFP 21, titled Higher-Order Probabilistic Adversarial Computations: Categorical Semantics and Program Logics. Adversarial computations are a widely studied class of computations where resource-bounded probabilistic adversaries have access to oracles, i.e., probabilistic procedures with private state. We develop program logics for reasoning about adversarial computations in a higher-order setting: unary program logics for reasoning about error probabilities and expected values, and a relational logic for reasoning about coupling-based properties. All logics feature rules for adversarial computations, and yield guarantees that are valid for all adversaries that satisfy a fixed resource policy. We prove the soundness of the logics in the category of quasi-Borel spaces, using a general notion of graded predicate liftings, and we use logical relations over graded predicate liftings to establish the soundness of proof rules for adversaries.
MP4 File (3473598.mp4)
Presentation Videos

References

[1]
Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Pierre-Yves Strub. 2017. A relational logic for higher-order programs. PACMPL, 1, ICFP (2017), 21:1–21:29. https://doi.org/10.1145/3110265
[2]
Alejandro Aguirre, Gilles Barthe, Justin Hsu, Benjamin Lucien Kaminski, Joost-Pieter Katoen, and Christoph Matheja. 2021. A Pre-Expectation Calculus for Probabilistic Sensitivity. Proc. ACM Program. Lang., 5, POPL (2021), Article 52, Jan., 28 pages. https://doi.org/10.1145/3434333
[3]
Alejandro Aguirre and Shin-ya Katsumata. 2020. Weakest Preconditions in Fibrations. In 36th Mathematical Foundations of Programming Semantics Conference, MFPS 2020 (Electronic Notes in Theoretical Computer Science, Vol. 352). 5–27. issn:1571-0661 https://doi.org/10.1016/j.entcs.2020.09.002
[4]
Robert J Aumann. 1961. Borel structures for function spaces. Illinois Journal of Mathematics, 5, 4 (1961), 614–630. https://doi.org/10.1215/ijm/1255631584
[5]
Manuel Barbosa, Gilles Barthe, Benjamin Grégoire, Adrien Koutsos, and Pierre-Yves Strub. 2021. Mechanized Proofs of Adversarial Complexity and Application to Universal Composability. Cryptology ePrint Archive, Report 2021/156. https://eprint.iacr.org/2021/156
[6]
Gilles Barthe, Marion Daubignard, Bruce M. Kapron, and Yassine Lakhnech. 2010. Computational indistinguishability logic. In Proceedings of the 17th ACM Conference on Computer and Communications Security, CCS 2010, Chicago, Illinois, USA, October 4-8, 2010, Ehab Al-Shaer, Angelos D. Keromytis, and Vitaly Shmatikov (Eds.). ACM, 375–386. https://doi.org/10.1145/1866307.1866350
[7]
Gilles Barthe, Thomas Espitau, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2018. Proving expected sensitivity of probabilistic programs. Proc. ACM Program. Lang., 2, POPL (2018), 57:1–57:29. https://doi.org/10.1145/3158145
[8]
Gilles Barthe, Noémie Fong, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016. Advanced Probabilistic Couplings for Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, October 24-28, 2016, Edgar R. Weippl, Stefan Katzenbeisser, Christopher Kruegel, Andrew C. Myers, and Shai Halevi (Eds.). ACM, 55–67. https://doi.org/10.1145/2976749.2978391
[9]
Gilles Barthe, Cédric Fournet, Benjamin Grégoire, Pierre-Yves Strub, Nikhil Swamy, and Santiago Zanella Béguelin. 2014. Probabilistic relational verification for cryptographic implementations. In POPL 2014, Suresh Jagannathan and Peter Sewell (Eds.). https://doi.org/10.1145/2578855.2535847
[10]
Gilles Barthe, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, and Pierre-Yves Strub. 2016. A Program Logic for Union Bounds. In 43rd International Colloquium on Automata, Languages, and Programming, ICALP 2016, July 11-15, 2016, Rome, Italy, Ioannis Chatzigiannakis, Michael Mitzenmacher, Yuval Rabani, and Davide Sangiorgi (Eds.) (LIPIcs, Vol. 55). Schloss Dagstuhl - Leibniz-Zentrum fuer Informatik, 107:1–107:15. https://doi.org/10.4230/LIPIcs.ICALP.2016.107
[11]
Gilles Barthe, Benjamin Grégoire, and Santiago Zanella Béguelin. 2009. Formal certification of code-based cryptographic proofs. In POPL 2009, Savannah, GA, USA, January 21-23, 2009. https://doi.org/10.1145/1480881.1480894
[12]
Gilles Barthe, Boris Köpf, Laurent Mauborgne, and Martín Ochoa. 2014. Leakage Resilience against Concurrent Cache Attacks. In Proc. 3rd Conference on Principles of Security and Trust (POST ’14). Springer. https://doi.org/10.1007/978-3-642-54792-8_8
[13]
Gilles Barthe, Boris Köpf, Federico Olmedo, and Santiago Zanella Béguelin. 2012. Probabilistic relational reasoning for differential privacy. In POPL 2012, Philadelphia, Pennsylvania, USA, January 22-28, 2012. https://doi.org/10.1145/2103656.2103670
[14]
Nick Benton, Martin Hofmann, and Vivek Nigam. 2014. Abstract effects and proof-relevant logical relations. In The 41st Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL ’14, San Diego, CA, USA, January 20-21, 2014, Suresh Jagannathan and Peter Sewell (Eds.). ACM, 619–632. https://doi.org/10.1145/2535838.2535869
[15]
Ales Bizjak and Lars Birkedal. 2015. Step-Indexed Logical Relations for Probability. In FoSSaCS 2015, London, UK, April 11-18, 2015. Proceedings. https://doi.org/10.1007/978-3-662-46678-0_18
[16]
Burton H. Bloom. 1970. Space/Time Trade-offs in Hash Coding with Allowable Errors. Commun. ACM, 13, 7 (1970), 422–426. https://doi.org/10.1145/362686.362692
[17]
Chris Brzuska, Antoine Delignat-Lavaud, Cédric Fournet, Konrad Kohbrok, and Markulf Kohlweiss. 2018. State Separation for Code-Based Game-Playing Proofs. In Advances in Cryptology - ASIACRYPT 2018 - 24th International Conference on the Theory and Application of Cryptology and Information Security, Brisbane, QLD, Australia, December 2-6, 2018, Proceedings, Part III, Thomas Peyrin and Steven D. Galbraith (Eds.) (Lecture Notes in Computer Science, Vol. 11274). Springer, 222–249. https://doi.org/10.1007/978-3-030-03332-3_9
[18]
David Clayton, Christopher Patton, and Thomas Shrimpton. 2019. Probabilistic Data Structures in Adversarial Environments. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, CCS 2019, London, UK, November 11-15, 2019, Lorenzo Cavallaro, Johannes Kinder, XiaoFeng Wang, and Jonathan Katz (Eds.). ACM, 1317–1334. https://doi.org/10.1145/3319535.3354235
[19]
Raphaëlle Crubillé and Ugo Dal Lago. 2015. Metric Reasoning about λ -Terms: The Affine Case. In 30th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2015, Kyoto, Japan, July 6-10, 2015. IEEE Computer Society, 633–644. https://doi.org/10.1109/LICS.2015.64
[20]
Vincent Danos and Thomas Ehrhard. 2011. Probabilistic coherence spaces as a model of higher-order probabilistic computation. Inf. Comput., 209, 6 (2011), 966–991. https://doi.org/10.1016/j.ic.2011.02.001
[21]
Dominique Devriese, Lars Birkedal, and Frank Piessens. 2016. Reasoning about Object Capabilities with Logical Relations and Effect Parametricity. In IEEE European Symposium on Security and Privacy (EuroS&P). 147–162. https://doi.org/10.1109/EuroSP.2016.22
[22]
Cynthia Dwork and Aaron Roth. 2014. The Algorithmic Foundations of Differential Privacy. Foundations and Trends in Theoretical Computer Science, 9, 3–4 (2014), 211–407. http://dx.doi.org/10.1561/0400000042
[23]
Deepak Garg, Jason Franklin, Dilsun Kirli Kaynar, and Anupam Datta. 2010. Compositional System Security with Interface-Confined Adversaries. Electr. Notes Theor. Comput. Sci., 265 (2010), 49–71. https://doi.org/10.1016/j.entcs.2010.08.005
[24]
Thomas Gerbet, Amrit Kumar, and Cédric Lauradoux. 2015. The Power of Evil Choices in Bloom Filters. In 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2015, Rio de Janeiro, Brazil, June 22-25, 2015. IEEE Computer Society, 101–112. https://doi.org/10.1109/DSN.2015.21
[25]
Chris Heunen, Ohad Kammar, Sam Staton, and Hongseok Yang. 2017. A convenient category for higher-order probability theory. In 32nd Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2017, Reykjavik, Iceland, June 20-23, 2017. IEEE Computer Society, 1–12. https://doi.org/10.1109/LICS.2017.8005137
[26]
R. Impagliazzo and S. Rudich. 1989. Limits on the provable consequences of one-way permutations. In 21st Annual ACM Symposium on Theory of Computing, 1989. ACM, New York. 44–61. https://doi.org/10.1145/73007.73012
[27]
Kenneth E. Iverson. 1962. A Programming Language. John Wiley & Sons, Inc., USA. isbn:0471430145
[28]
Limin Jia, Shayak Sen, Deepak Garg, and Anupam Datta. 2015. A Logic of Programs with Interface-Confined Code. In IEEE 28th Computer Security Foundations Symposium (CSF). 512–525. https://doi.org/10.1109/CSF.2015.38
[29]
Ralf Jung, David Swasey, Filip Sieczkowski, Kasper Svendsen, Aaron Turon, Lars Birkedal, and Derek Dreyer. 2015. Iris: Monoids and Invariants as an Orthogonal Basis for Concurrent Reasoning. In Proceedings of the 42nd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 2015, Mumbai, India, January 15-17, 2015, Sriram K. Rajamani and David Walker (Eds.). ACM, 637–650. https://doi.org/10.1145/2676726.2676980
[30]
Benjamin Lucien Kaminski, Joost-Pieter Katoen, Christoph Matheja, and Federico Olmedo. 2016. Weakest Precondition Reasoning for Expected Run-Times of Probabilistic Programs. 9632, 364–389. https://doi.org/10.1007/978-3-662-49498-1_15
[31]
Shin-ya Katsumata. 2014. Parametric Effect Monads and Semantics of Effect Systems. In Proceedings of the 41st ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL ’14). Association for Computing Machinery, New York, NY, USA. 633–645. isbn:9781450325448 https://doi.org/10.1145/2535838.2535846
[32]
Shin-ya Katsumata, Tetsuya Sato, and Tarmo Uustalu. 2018. Codensity Lifting of Monads and its Dual. Log. Methods Comput. Sci., 14, 4 (2018), https://doi.org/10.23638/LMCS-14(4:6)2018
[33]
Dexter Kozen. 1985. A Probabilistic PDL. 30, 2 (1985), 162–178. https://doi.org/10.1145/800061.808758
[34]
Saunders MacLane. 1971. Categories for the Working Mathematician. Springer-Verlag, New York. Graduate Texts in Mathematics, Vol. 5.
[35]
Kenji Maillard, Catalin Hritcu, Exequiel Rivas, and Antoine Van Muylder. 2020. The next 700 relational program logics. Proc. ACM Program. Lang., 4, POPL (2020), 4:1–4:33. https://doi.org/10.1145/3371072
[36]
Cristina Matache and Sam Staton. 2019. A Sound and Complete Logic for Algebraic Effects. In Foundations of Software Science and Computation Structures - 22nd International Conference, FOSSACS 2019, Held as Part of the European Joint Conferences on Theory and Practice of Software, ETAPS 2019, Prague, Czech Republic, April 6-11, 2019, Proceedings, Mikolaj Bojanczyk and Alex Simpson (Eds.) (Lecture Notes in Computer Science, Vol. 11425). Springer, 382–399. https://doi.org/10.1007/978-3-030-17127-8_22
[37]
Carroll Morgan, Annabelle McIver, and Karen Seidel. 1996. Probabilistic Predicate Transformers. 18, 3 (1996), 325–353. https://doi.org/10.1145/229542.229547
[38]
Aleksandar Nanevski, Anindya Banerjee, and Deepak Garg. 2013. Dependent Type Theory for Verification of Information Flow and Access Control Policies. ACM Trans. Program. Lang. Syst., 35, 2 (2013), 6:1–6:41. https://doi.org/10.1145/2491522.2491523
[39]
Aleksandar Nanevski, J. Gregory Morrisett, and Lars Birkedal. 2008. Hoare type theory, polymorphism and separation. J. Funct. Program., 18, 5-6 (2008), 865–911. https://doi.org/10.1017/S0956796808006953
[40]
Moni Naor and Eylon Yogev. 2019. Bloom Filters in Adversarial Environments. ACM Trans. Algorithms, 15, 3 (2019), 35:1–35:30. https://doi.org/10.1145/3306193
[41]
Andrew Pitts and Ian Stark. 1998. Operational Reasoning for Functions with Local State. In Higher Order Operational Techniques in Semantics, Andrew Gordon and Andrew Pitts (Eds.). Publications of the Newton Institute, Cambridge University Press, 227–273. http://www.inf.ed.ac.uk/~stark/operfl.html
[42]
Gordon Plotkin. 1973. Lambda-definability and logical relations.
[43]
Thomas Ristenpart, Hovav Shacham, and Thomas Shrimpton. 2011. Careful with Composition: Limitations of the Indifferentiability Framework. In Advances in Cryptology - EUROCRYPT 2011 - 30th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Tallinn, Estonia, May 15-19, 2011. Proceedings, Kenneth G. Paterson (Ed.) (Lecture Notes in Computer Science, Vol. 6632). Springer, 487–506. https://doi.org/10.1007/978-3-642-20465-4_27
[44]
Tetsuya Sato. 2016. Approximate Relational Hoare Logic for Continuous Random Samplings. In The Thirty-second Conference on the Mathematical Foundations of Programming Semantics, MFPS 2016, Carnegie Mellon University, Pittsburgh, PA, USA, May 23-26, 2016, Lars Birkedal (Ed.) (Electronic Notes in Theoretical Computer Science, Vol. 325). Elsevier, 277–298. https://doi.org/10.1016/j.entcs.2016.09.043
[45]
Tetsuya Sato, Alejandro Aguirre, Gilles Barthe, Marco Gaboardi, Deepak Garg, and Justin Hsu. 2019. Formal verification of higher-order probabilistic programs: reasoning about approximation, convergence, Bayesian inference, and optimization. PACMPL, 3, POPL (2019), 38:1–38:30. https://doi.org/10.1145/3290351
[46]
Adam Ścibior, Ohad Kammar, Matthijs Vákár, Sam Staton, Hongseok Yang, Yufei Cai, Klaus Ostermann, Sean K. Moss, Chris Heunen, and Zoubin Ghahramani. 2017. Denotational Validation of Higher-order Bayesian Inference. Proc. ACM Program. Lang., 2, POPL (2017), Article 60, Dec., 29 pages. issn:2475-1421 https://doi.org/10.1145/3158148
[47]
Joseph Tassarotti and Robert Harper. 2019. A Separation Logic for Concurrent Randomized Programs. Proc. ACM Program. Lang., 3, POPL (2019), Article 64, Jan., 30 pages. https://doi.org/10.1145/3290377
[48]
Matthijs Vákár, Ohad Kammar, and Sam Staton. 2019. A domain theory for statistical probabilistic programming. PACMPL, 3, POPL (2019), 36:1–36:29. https://doi.org/10.1145/3290349

Cited By

View all

Index Terms

  1. Higher-order probabilistic adversarial computations: categorical semantics and program logics

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Proceedings of the ACM on Programming Languages
    Proceedings of the ACM on Programming Languages  Volume 5, Issue ICFP
    August 2021
    1011 pages
    EISSN:2475-1421
    DOI:10.1145/3482883
    Issue’s Table of Contents
    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 August 2021
    Published in PACMPL Volume 5, Issue ICFP

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Probabilistic programming
    2. program logics
    3. semantic models

    Qualifiers

    • Research-article

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)128
    • Downloads (Last 6 weeks)14
    Reflects downloads up to 09 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Get Access

    Login options

    Full Access

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media