Validating Traces of Distributed Programs Against TLA+ Specifications
Pages 126 - 143
Abstract
TLA+ is a formal language for specifying systems, including distributed algorithms, that is supported by powerful verification tools. In this work we present a framework for checking if traces of distributed programs are compatible with high-level specifications written in TLA+. The problem is reduced to a constrained model checking problem, realized using the TLC model checker. Our framework consists of an API for instrumenting Java programs in order to record traces of executions, of a collection of TLA+ operators that are used for relating those traces to specifications, and of scripts for running the model checker. Crucially, traces only contain updates to specification variables rather than full values, and developers may choose to trace only certain variables. We have applied our approach to several distributed programs, detecting discrepancies between the specifications and the implementations in all cases. We discuss reasons for these discrepancies, best practices for instrumenting programs, and how to interpret the verdict produced by TLC.
References
[1]
Cirstea, H., Kuppe, M.A., Loillier, B., Merz, S.: Validating traces of distributed programs against TLA specifications. arXiv:2404.16075 [cs.DC] (2024)
[2]
Cousineau, D., Doligez, D., Lamport, L., Merz, S., Ricketts, D., Vanzetto, H.: TLA Proofs. In: Giannakopoulou, D., Méry, D. (eds.) FM 2012. LNCS, vol. 7436, pp. 147–154. Springer, Heidelberg (2012).
[3]
Davis AJJ, Hirschhorn M, and Schvimer J eXtreme modelling in practice Proc. VLDB Endowment 2020 13 9 1346-1358
[4]
Dijkstra, E.W.: EWD 998: Shmuel Safra’s version of termination detection. http://www.cs.utexas.edu/users/EWD/ewd09xx/EWD998.PDF (1987)
[5]
etcd project. TLA+ specification and trace validation for raft library: a brief guide. https://github.com/etcd-io/raft/tree/main/tla (2024)
[6]
Falcone Y, Havelund K, and Reger G A tutorial on runtime verification Eng. Depend. Softw. Syst. 2013 34 141-175
[7]
Fekete, A.: Snapshot Isolation. In: Liu, L., Özsu, M.T. (eds.) Encyclopedia of Database Systems, pp. 2659–2664. Springer US, Boston, MA (2009).
[8]
Foo, D., Costea, A., Chin, W.-N.: Protocol conformance with choreographic PlusCal. In: David, C., Sun, M. (eds.) Theoretical Aspects of Software Engineering: 17th International Symposium, TASE 2023, Bristol, UK, July 4–6, 2023, Proceedings, pp. 126–145. Springer Nature Switzerland, Cham (2023).
[9]
Hackett, F., Hosseini, S., Costa, R., Do, M., Beschastnikh, I.: Compiling distributed system models with PGo. In: Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, vol. 2, pp. 159–175, Vancouver BC Canada. ACM (2023)
[10]
Havelund K Havelund K, Penix J, and Visser W Using Runtime Analysis to Guide Model Checking of Java Programs SPIN Model Checking and Software Verification 2000 Heidelberg Springer 245-264
[11]
Howard H et al. Confidential Consortium Framework: secure multiparty applications with confidentiality, integrity, and high availability Proc. VLDB Endowment 2023 17 2 225-240
[12]
Howard, H., Kuppe, M.A., Ashton, E., Chamayou, A., Crooks, N.: Smart casual verification of CCF’s distributed consensus and consistency protocols. arXiv:2406.17455 [cs.DC] (2024)
[13]
Howard, Y., Gruner, S., Gravell, A., Ferreira, C., Augusto, J.C.: Model-based trace-checking. arXiv:1111.2825 [cs] (2011)
[14]
Konnov, I., Kukovec, J., Tran, T.-H.: TLA+ model checking made symbolic. Proc. ACM Program. Lang., 3(OOPSLA) (2019)
[15]
Konnov, I., Kuppe, M., Merz, S.: Specification and verification with the TLA+ Trifecta: TLC, Apalache, and TLAPS. In: Margaria, T., Steffen, B. (eds.) Leveraging Applications of Formal Methods, Verification and Validation. Verification Principles: 11th International Symposium, ISoLA 2022, Rhodes, Greece, October 22–30, 2022, Proceedings, Part I, pp. 88–105. Springer International Publishing, Cham (2022).
[16]
Kuppe, M.A.: Implementing a TLA specification: EWD998Chan. https://github.com/tlaplus/Examples/pull/75 (2023)
[17]
Kuppe, M.A.: The TLA debugger. In: Masci, P., Bernardeschi, C., Graziani, P., Koddenbrock, M., Palmieri, M., editors, Software Engineering and Formal Methods. SEFM 2022 Co-located Workshops, vol. 13765, pp. 174–180. Springer, Cham (2023).
[18]
Lamport L Specifying Systems 2002 Addison-Wesley, Boston Mass
[19]
Lamport, L., et al.: TLA examples. https://github.com/tlaplus/examples/
[20]
Newcombe C, Rath T, Zhang F, Munteanu B, Brooker M, and Deardeuff M How Amazon Web Services uses formal methods Commun. ACM 2015 58 4 66-73
[21]
Niu, Z., Dong, L., Zhu, Y., Chen, L.: Verifying zookeeper based on model-based runtime trace-checking using TLA. In: Proceedings of the 7th International Conference on Cyber Security and Information Engineering, pp. 13–18, Brisbane QLD Australia. ACM (2022)
[22]
Ongaro, D., Ousterhout, J.: In search of an understandable consensus algorithm. In: 2014 USENIX Annual Technical Conference, pp. 305–319, Philadelphia, PA (2014). USENIX Association
[23]
Pressler, R.: Conjunction capers: a TLA truffle. https://conf.tlapl.us/2020/ (2020)
[24]
Schultz, W., Zhou, S., Dardik, I., Tripakis, S.: Design and analysis of a logless dynamic reconfiguration protocol. In: Bramas, Q., Gramoli, V., Milani, A., editors 25th International Conference Principles of Distributed Systems (OPODIS 2021), vol. 217 of LIPIcs, pp. 26:1-26:16, Strasbourg, France (2021). Schloss Dagstuhl - Leibniz-Zentrum für Informatik
[25]
Tasiran, S., Yu, Y., Batson, B., Kreider, S.: Using formal specifications to monitor and guide simulation: verifying the cache coherence engine of the Alpha 21364 microprocessor. In: Proceedings of the 3rd IEEE Workshop on Microprocessor Test and Verification, Common Challenges and Solutions (2002)
[26]
Tretmans J Test generation with inputs, outputs and repetitive quiescence Softw. Concepts Tools 1996 17 3 103-120
[27]
Vaandrager FW Model learning Commun. ACM 2017 60 2 86-95
[28]
Wang, D., Dou, W., Gao, Y., Wu, C., Wei, J., Huang, T.: Model checking guided testing for distributed systems. In: Proceedings of the Eighteenth European Conference on Computer Systems, pp. 127–143, Rome Italy. ACM (2023)
[29]
Yu Y, Manolios P, and Lamport L Pierre L and Kropf T Model Checking TLA+ Specifications Correct Hardware Design and Verification Methods 1999 Heidelberg Springer 54-66
Index Terms
- Validating Traces of Distributed Programs Against TLA+ Specifications
Index terms have been assigned to the content through auto-classification.
Recommendations
Validating feature-based specifications: Research Articles
It is argued that specifications should be rigorously validated against requirements. This is useful to build confidence in a specification and to check a specification after it or the requirements have changed. The multiple-use scenario test and ...
Model Checking TLA+ Specifications
CHARME '99: Proceedings of the 10th IFIP WG 10.5 Advanced Research Working Conference on Correct Hardware Design and Verification MethodsTLA+ is a specification language for concurrent and reactive systems that combines the temporal logic TLA with full first-order logic and ZF set theory. TLC is a new model checker for debugging a TLA+ specification by checking invariance properties of a ...
Comments
Information & Contributors
Information
Published In
Nov 2024
473 pages
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 26 November 2024
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025