research-article

CheckINN: Wide Range Neural Network Verification in Imandra

Authors:

Remi Desmartin,

Grant Passmore,

Ekaterina Komendantskaya,

Matthew DaggitAuthors Info & Claims

PPDP '22: Proceedings of the 24th International Symposium on Principles and Practice of Declarative Programming

Article No.: 3, Pages 1 - 14

https://doi.org/10.1145/3551357.3551372

Published: 20 September 2022 Publication History

Abstract

Neural networks are increasingly relied upon as components of complex safety-critical systems such as autonomous vehicles. There is high demand for tools and methods that embed neural network verification in a larger verification cycle. However, neural network verification is difficult due to a wide range of verification properties of interest, each typically only amenable to verification in specialised solvers. In this paper, we show how Imandra, a functional programming language and a theorem prover originally designed for verification, validation and simulation of financial infrastructure can offer a holistic infrastructure for neural network verification. We develop a novel library CheckINN that formalises neural networks in Imandra, and covers different important facets of neural network verification.

References

[1]

Edward W. Ayers, Francisco Eiras, Majd Hawasly, and Iain Whiteside. 2020. PaRoT: A Practical Framework for Robust Deep Neural Network Training. In NASA Formal Methods - 12th International Symposium, NFM 2020, Moffett Field, CA, USA, May 11-15, 2020, Proceedings(LNCS, Vol. 12229). Springer, 63–84.

[2]

A. Bagnall and G. Stewart. 2019. Certifying True Error: Machine Learning in Coq with Verified Generalisation Guarantees. AAAI (2019).

[3]

R. Boyer and J. Moore. 1979. A Computational Logic. ACM Monograph Series. Academic Press, New York.

[4]

Robert S. Boyer and J Strother Moore. 1988. Integrating decision procedures into heuristic theorem provers: a case study of linear arithmetic. Machine intelligence(1988), 83–124.

[5]

Marco Casadio, Ekaterina Komendantskaya, Matthew L. Daggitt, Wen Kokke, Guy Katz, Guy Amir, and Idan Refaeli. 2022. Neural Network Robustness as a Verification Property: A Principled Case Study. In Computer Aided Verification (CAV 2022)(Lecture Notes in Computer Science). Springer.

[6]

François Chollet 2015. Keras. https://keras.io.

[7]

Koen Claessen and John Hughes. 2000. QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs. In Proceedings of the Fifth ACM SIGPLAN International Conference on Functional Programming(ICFP ’00). Association for Computing Machinery, New York, NY, USA, 268–279. https://doi.org/10.1145/351240.351266

Digital Library

[8]

Leonardo de Moura and Grant Olney Passmore. 2013. Computation in Real Closed Infinitesimal and Transcendental Extensions of the Rationals. In CADE.

[9]

Remi Desmartin, Grant Passmore, Ekaterina Kmendantskaya, and Matthew L. Daggitt. 2022. CNN Library in Imandra. https://github.com/aisec-private/ImandraNN.

[10]

Remi Desmartin, Grant Passmore, and Ekaterina Komendantskaya. 2022. Neural Networks in Imandra: Matrix Representation as a Verification Choice. https://arxiv.org/abs/2205.09556.

[11]

Remi Desmartin, Grant Passmore, Ekaterina Komendantskaya, and Matthew Daggitt. 2022. CheckINN: Wide Range Neural Network Verification in Imandra (Extended). https://doi.org/10.48550/arXiv.2207.10562 arXiv:2207.10562 [cs].

[12]

Kirsty Duncan, Ekaterina Komendantskaya, Robert J. Stewart, and Michael A. Lones. 2020. Relative Robustness of Quantized Neural Networks Against Adversarial Attacks. In 2020 International Joint Conference on Neural Networks, IJCNN 2020, Glasgow, United Kingdom, July 19-24, 2020. 1–8. https://doi.org/10.1109/IJCNN48605.2020.9207596

[13]

Bruno Dutertre and Leonardo de Moura. 2006. A Fast Linear-Arithmetic Solver for DPLL(T). In Computer Aided Verification. Springer Berlin Heidelberg, 81–94.

[14]

Marc Fischer, Mislav Balunovic, Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, and Martin T. Vechev. 2019. DL2: Training and Querying Neural Networks with Logic. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9-15 June 2019, Long Beach, California, USA(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 1931–1941. http://proceedings.mlr.press/v97/fischer19a.html

[15]

T. Gehr, M. Mirman, D. Drachsler-Cohen, E. Tsankov, S. Chaudhuri, and M. Vechev. 2018. AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation. In S&P.

[16]

Dan R. Ghica and Todd Waugh Ambridge. 2021. Global Optimisation with Constructive Reals. In 36th Annual ACM/IEEE Symposium on Logic in Computer Science, LICS 2021, Rome, Italy, June 29 - July 2, 2021. 1–13.

[17]

P. W. Grant, J. A. Sharp, M. F. Webster, and X. Zhang. 1996. Sparse matrix representations in a functional language. Journal of Functional Programming 6, 1 (Jan. 1996), 143–170. https://doi.org/10.1017/S095679680000160X Publisher: Cambridge University Press.

[18]

Jónathan Heras, María Poza, Maxime Dénès, and Laurence Rideau. 2011. Incidence Simplicial Matrices Formalized in Coq/SSReflect. In Intelligent Computer Mathematics(Lecture Notes in Computer Science), James H. Davenport, William M. Farmer, Josef Urban, and Florian Rabe (Eds.). Springer, Berlin, Heidelberg, 30–44. https://doi.org/10.1007/978-3-642-22673-1_3

[19]

Xiaowei Huang, Marta Kwiatkowska, Sen Wang, and Min Wu. 2017. Safety Verification of Deep Neural Networks. In Computer Aided Verification - 29th International Conference, CAV 2017, Heidelberg, Germany, July 24-28, 2017, Proceedings, Part I(Lecture Notes in Computer Science, Vol. 10426). 3–29.

[20]

Kai Jia and Martin Rinard. 2021. Exploiting Verified Neural Networks via Floating Point Numerical Error. In Static Analysis - 28th International Symposium, SAS 2021, Chicago, IL, USA, October 17-19, 2021, Proceedings(Lecture Notes in Computer Science, Vol. 12913). Springer, 191–205.

[21]

Dejan Jovanović and Leonardo de Moura. 2013. Solving non-linear arithmetic. ACM Communications in Computer Algebra 46, 3/4 (Jan. 2013), 104.

Digital Library

[22]

G. Katz, C. Barrett, D. Dill, K. Julian, and M. Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. In CAV.

[23]

Guy Katz, Clark Barrett, David Dill, Kyle Julian, and Mykel Kochenderfer. 2017. Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks. arXiv:1702.01135 [cs] (May 2017). http://arxiv.org/abs/1702.01135 arXiv:1702.01135.

[24]

Guy Katz, Derek A. Huang, Duligur Ibeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljic, David L. Dill, Mykel J. Kochenderfer, and Clark W. Barrett. 2019. The Marabou Framework for Verification and Analysis of Deep Neural Networks. In CAV 2019, Part I(LNCS, Vol. 11561). Springer, 443–452.

[25]

Wen Kokke, Ekaterina Komendantskaya, Daniel Kienitz, Robert Atkey, and David Aspinall. 2020. Neural Networks, Secure by Construction - An Exploration of Refinement Types. In Programming Languages and Systems - 18th Asian Symposium, APLAS 2020, Fukuoka, Japan, November 30 - December 2, 2020, Proceedings(Lecture Notes in Computer Science, Vol. 12470). Springer, 67–85.

[26]

Alexander Kozlov, Ivan Lazarevich, Vasily Shamporov, Nikolay Lyalyushkin, and Yury Gorbachev. 2021. Neural Network Compression Framework for Fast Model Inference. In Intelligent Computing, Kohei Arai (Ed.). Springer International Publishing, Cham, 213–232.

[27]

Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. CoRR abs/1806.08342(2018). arXiv:1806.08342http://arxiv.org/abs/1806.08342

[28]

Ori Lahav and Guy Katz. 2021. Pruning and Slicing Neural Networks using Formal Verification. In Formal Methods in Computer Aided Design, FMCAD 2021, New Haven, CT, USA, October 19-22, 2021. 1–10. https://doi.org/10.34727/2021/isbn.978-3-85448-046-4_27

[29]

Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards Deep Learning Models Resistant to Adversarial Attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https://openreview.net/forum?id=rJzIBfZAb

[30]

Elisabetta De Maria, Abdorrahim Bahrami, Thibaud L’Yvonnet, Amy P. Felty, Daniel Gaffé, Annie Ressouche, and Franck Grammont. 2022. On the use of formal methods to model and verify neuronal archetypes. Frontiers Comput. Sci. 16, 3 (2022), 163404.

Digital Library

[31]

Grant Olney Passmore. 2021. Some Lessons Learned in the Industrialization of Formal Methods for Financial Algorithms. In Formal Methods - 24th International Symposium, FM 2021, Virtual Event, November 20-26, 2021, Proceedings(Lecture Notes in Computer Science, Vol. 13047). Springer, 717–721.

[32]

Grant O. Passmore, Simon Cruanes, Denis Ignatovich, Dave Aitken, Matt Bray, Elijah Kagan, Kostya Kanishev, Ewen Maclean, and Nicola Mometto. 2020. The Imandra Automated Reasoning System (System Description). In Automated Reasoning - 10th International Joint Conference, IJCAR 2020, Paris, France, July 1-4, 2020, Proceedings, Part II, Vol. 12167. Springer, 464–471.

[33]

Connor Shorten and Taghi M. Khoshgoftaar. 2019. A survey on Image Data Augmentation for Deep Learning. Journal of Big Data 6, 1 (July 2019). https://doi.org/10.1186/s40537-019-0197-0 Publisher: Springer Science and Business Media LLC.

[34]

Joseph Sill. 1998. Monotonic Networks. California Institute of Technology.

[35]

Gagandeep Singh, Timon Gehr, Markus Püschel, and Martin T. Vechev. 2019. An abstract domain for certifying neural networks. PACMPL 3, POPL (2019), 41:1–41:30. https://doi.org/10.1145/3290354

Digital Library

[36]

Natalia Slusarz, Ekaterina Komendantskaya, Matthew L. Daggitt, and Robert Stewart. 2022. Differentiable Logics for Neural Network Training and Verification. https://doi.org/10.48550/arXiv.2207.06741 arXiv:2207.06741 [cs].

[37]

Joseph Tassarotti, Koundinya Vajjha, Anindya Banerjee, and Jean-Baptiste Tristan. 2021. A formal proof of PAC learnability for decision stumps. In CPP ’21: 10th ACM SIGPLAN International Conference on Certified Programs and Proofs, Virtual Event, Denmark, January 17-19, 2021. ACM, 5–17.

Digital Library

[38]

Antoine Wehenkel and Gilles Louppe. 2019. Unconstrained Monotonic Neural Networks. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada. 1543–1553.

[39]

James Wood. 2019. Vectors and Matrices in Agda. https://personal.cis.strath.ac.uk/james.wood.100/blog/html/VecMat.html

Cited By

Molnár VGraics BVörös ATonetta SCristoforetti LKimberly GDyer PGiammarco KKoethe MHester JSmith JGrimm CCombemale BWimmer MChechik MEgyed A(2024)Towards the Formal Verification of SysML v2 ModelsProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems10.1145/3652620.3687820(1086-1095)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3652620.3687820

Index Terms

CheckINN: Wide Range Neural Network Verification in Imandra
1. Theory of computation
  1. Logic
    1. Higher order logic
    2. Logic and verification
  2. Semantics and reasoning
    1. Program reasoning
      1. Program verification

Recommendations

Approximate Conformance Verification of Deep Neural Networks
NASA Formal Methods
Abstract
We consider the problem of approximate conformance checking on deep neural networks. More precisely, given two neural networks and a conformance bound $ϵ$ , we need to check if the neural network outputs are within $ϵ$ given the same inputs from the ...
NNV 2.0: The Neural Network Verification Tool
Computer Aided Verification
Abstract
This manuscript presents the updated version of the Neural Network Verification (NNV) tool. NNV is a formal verification software tool for deep learning models and cyber-physical systems with neural network components. NNV was first introduced as ...
Neural Network Verification Using Residual Reasoning
Software Engineering and Formal Methods
Abstract
With the increasing integration of neural networks as components in mission-critical systems, there is an increasing need to ensure that they satisfy various safety and liveness requirements. In recent years, numerous sound and complete ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PPDP '22: Proceedings of the 24th International Symposium on Principles and Practice of Declarative Programming

September 2022

187 pages

ISBN:9781450397032

DOI:10.1145/3551357

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 September 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

PPDP 2022

PPDP 2022: 24th International Symposium on Principles and Practice of Declarative Programming

September 20 - 22, 2022

Tbilisi, Georgia

Acceptance Rates

Overall Acceptance Rate 230 of 486 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
34
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Molnár VGraics BVörös ATonetta SCristoforetti LKimberly GDyer PGiammarco KKoethe MHester JSmith JGrimm CCombemale BWimmer MChechik MEgyed A(2024)Towards the Formal Verification of SysML v2 ModelsProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems10.1145/3652620.3687820(1086-1095)Online publication date: 22-Sep-2024
https://dl.acm.org/doi/10.1145/3652620.3687820

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents