article

The first reactive synthesis competition (SYNTCOMP 2014)

Authors:

Roderick Bloem,

Romain Brenguier,

Rüdiger Ehlers,

Timotheus Hell,

Robert Könighofer,

Guillermo A. Pérez,

Jean-François Raskin,

Leander Tentrup,

Adam WalkerAuthors Info & Claims

International Journal on Software Tools for Technology Transfer (STTT), Volume 19, Issue 3

Pages 367 - 390

https://doi.org/10.1007/s10009-016-0416-3

Published: 01 June 2017 Publication History

Abstract

We introduce the reactive synthesis competition (SYNTCOMP), a long-term effort intended to stimulate and guide advances in the design and application of synthesis procedures for reactive systems. The first iteration of SYNTCOMP is based on the controller synthesis problem for finite-state systems and safety specifications. We provide an overview of this problem and existing approaches to solve it, and report on the design and results of the first SYNTCOMP. This includes the definition of the benchmark format, the collection of benchmarks, the rules of the competition, and the five synthesis tools that participated. We present and analyze the results of the competition and draw conclusions on the state of the art. Finally, we give an outlook on future directions of SYNTCOMP.

References

[1]

Alur, R., Bodík, R., Dallal, E., Fisman, D., Garg, P., Juniwal, G., Kress-Gazit, H., Madhusudan, P., Martin, M.M.K., Raghothaman, M., Saha, S., Seshia, S.A., Singh, R., Solar-Lezama, A., Torlak, E., Udupa, A.: Syntax-guided synthesis. In: Dependable Software Systems Engineering, NATO Science for Peace and Security Series, D: Information and Communication Security, vol. 40, pp. 1---25. IOS Press (2015)

[2]

Alur, R., Madhusudan, P., Nam, W.: Symbolic computational techniques for solving games. STTT 7(2), 118---128 (2005)

Digital Library

[3]

Aziz, A., Tasiran, S., Brayton, R.K.: BDD variable ordering for interacting finite state machines. In: DAC, pp. 283---288 (1994)

Digital Library

[4]

Balint, A., Diepold, D., Gall, D., Gerber, S., Kapler, G., Retz, R.: EDACC - an advanced platform for the experiment design, administration and analysis of empirical algorithms. In: LION 5. Selected Papers, LNCS, vol. 6683, pp. 586---599. Springer, (2011)

Digital Library

[5]

Barrett, C.W., de Moura, L.M., Stump, A.: Design and results of the first satisfiability modulo theories competition (SMT-COMP 2005). J. Autom. Reason. 35(4), 373---390 (2005)

Digital Library

[6]

Beyer, D.: Competition on software verification - (SV-COMP). In: TACAS, LNCS, vol. 7214, pp. 504---524. Springer (2012)

Digital Library

[7]

Beyer, D., Löwe, S., Wendler, P.: Benchmarking and resource measurement. In: SPIN 2015, LNCS, vol. 9232, pp. 160---178. Springer (2015)

Digital Library

[8]

Bloem, R., Cimatti, A., Greimel, K., Hofferek, G., Könighofer, R., Roveri, M., Schuppan, V., Seeber, R.: RATSY - A new requirements analysis tool with synthesis. In: CAV, LNCS, vol. 6174, pp. 425---429. Springer (2010)

Digital Library

[9]

Bloem, R., Egly, U., Klampfl, P., Könighofer, R., Lonsing, F.: SAT-based methods for circuit synthesis. In: FMCAD'14, pp. 31---34. IEEE (2014)

Digital Library

[10]

Bloem, R., Galler, S.J., Jobstmann, B., Piterman, N., Pnueli, A., Weiglhofer, M.: Automatic hardware synthesis from specifications: a case study. In: DATE, pp. 1188---1193. ACM (2007)

Digital Library

[11]

Bloem, R., Galler, S.J., Jobstmann, B., Piterman, N., Pnueli, A., Weiglhofer, M.: Specify, compile, run: hardware from PSL. Electr. Notes Theor. Comput. Sci. 190(4), 3---16 (2007)

Digital Library

[12]

Bloem, R., Jobstmann, B., Piterman, N., Pnueli, A., Sa'ar, Y.: Synthesis of reactive(1) designs. J. Comput. Syst. Sci. 78(3), 911---938 (2012)

Digital Library

[13]

Bloem, R., Könighofer, R., Seidl, M.: SAT-based synthesis methods for safety specs. In: VMCAI, LNCS, vol. 8318, pp. 1---20. Springer (2014)

Digital Library

[14]

Bohy, A., Bruyère, V., Filiot, E., Jin, N., Raskin, J.-F.: Acacia+, a tool for LTL synthesis. In: CAV, LNCS, vol. 7358, pp. 652---657. Springer (2012)

Digital Library

[15]

Bradley, A.R.: SAT-based model checking without unrolling. In: VMCAI, LNCS, vol. 6538, pp. 70---87. Springer (2011)

Digital Library

[16]

Brayton, R.K., Hachtel, G.D., Sangiovanni-Vincentelli, A.L., Somenzi, F., Aziz, A., Cheng, S., Edwards, S.A., Khatri, S.P., Kukimoto, Y., Pardo, A., Qadeer, S., Ranjan, R.K., Sarwary, S., Shiple, T.R., Swamy, G., Villa, T.: VIS: a system for verification and synthesis. In: CAV, LNCS, vol. 1102, pp. 428---432. Springer (1996)

Digital Library

[17]

Brayton, R.K., Mishchenko, A.: ABC: an academic industrial-strength verification tool. In: CAV, LNCS, vol. 6174, pp. 24---40. Springer (2010)

Digital Library

[18]

Brenguier, R., Pérez, G.A., Raskin, J.-F., Sankur, O.: AbsSynthe: abstract synthesis from succinct safety specifications. In: SYNT, EPTCS, vol. 157, pp. 100---116. Open Publishing Association (2014)

[19]

Bryant, R.E.: Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. 35(8), 677---691 (1986)

Digital Library

[20]

Büchi, J., Landweber, L.: Solving sequential conditions by finite-state strategies. Trans. Am. Math. Soc. 138, 295---311 (1969)

[21]

Burch, J.R., Clarke, E.M., Long, D.E.: Symbolic model checking with partitioned transistion relations. In: VLSI, pp. 49---58 (1991)

[22]

Chiang, T., Jiang. J.R.: Property-directed synthesis of reactive systems from safety specifications. In: ICCAD, pp. 794---801. ACM (2015)

Digital Library

[23]

Church, A.: Logic, arithmetic and automata. In: Proceedings of the International Congress of Mathematicians, pp. 23---35 (1962)

[24]

Coudert, O., Madre, J.C.: A unified framework for the formal verification of sequential circuits. In: ICCAD, pp. 126---129 (1990)

[25]

Cousot, P., Cousot, R.: Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In: POPL, pp. 238---252. ACM (1977)

Digital Library

[26]

de Alfaro, L., Roy, P.: Solving games via three-valued abstraction refinement. In: CONCUR, LNCS, vol. 4703, pp. 74---89. Springer (2007)

Digital Library

[27]

Ehlers, R.: Experimental aspects of synthesis. In: iWIGP, EPTCS, vol. 50, pp. 1---16 (2011)

[28]

Ehlers, R.: Unbeast: symbolic bounded synthesis. In: TACAS, LNCS, vol. 6605, pp. 272---275. Springer (2011)

Digital Library

[29]

Ehlers, R.: Symbolic bounded synthesis. Formal Methods Syst. Des. 40(2), 232---262 (2012)

Digital Library

[30]

Ehlers, R., Könighofer, R., Hofferek, G.: Symbolically synthesizing small circuits. In: FMCAD'12, pp. 91---100. IEEE (2012)

[31]

Emerson, E.A., Clarke, E.M.: Using branching time temporal logic to synthesize synchronization skeletons. Sci. Comput. Program. 2(3), 241---266 (1982)

[32]

Filiot, E., Jin, N., Raskin, J.: Exploiting structure in LTL synthesis. STTT 15(5---6), 541---561 (2013)

[33]

Filiot, E., Jin, N., Raskin, J.-F.: Antichains and compositional algorithms for LTL synthesis. Formal Methods Syst. Des. 39(3), 261---296 (2011)

Digital Library

[34]

Finkbeiner, B., Jacobs, S.: Lazy synthesis. In: VMCAI, LNCS, vol. 7148, pp. 219---234. Springer (2012)

Digital Library

[35]

Finkbeiner, B., Schewe, S.: Bounded synthesis. STTT 15(5---6), 519---539 (2013)

[36]

Graf, S., Saïdi, H.: Construction of abstract state graphs with PVS. In: CAV, LNCS, vol. 1254, pp. 72---83. Springer (1997)

Digital Library

[37]

Henzinger, T.A., Jhala, R., Majumdar, R.: Counterexample-guided control. In: ICALP, LNCS, vol. 2719, pp. 886---902, Springer (2003)

Digital Library

[38]

Hong, Y., Beerel, P.A., Burch, J.R., McMillan, K.L.: Sibling-substitution-based BDD minimization using don't cares. IEEE Trans. CAD of Integr. Circuits Syst. 19(1), 44---55 (2000)

Digital Library

[39]

Jacobs, S.: Extended AIGER format for synthesis. CoRR (2014). arXiv:1405.5793. Accessed Feb 2016

[40]

Jacobs, S., Bloem, R., Brenguier, R., Ehlers, R., Hell, T., Könighofer, R., Pérez, G.A., Raskin, J., Ryzhyk, L., Sankur, O., Seidl, M., Tentrup, L., Walker, A.: The first reactive synthesis competition (SYNTCOMP 2014). CoRR (2015). arXiv:1506.08726. Accessed Feb 2016

Digital Library

[41]

Jacobs, S., Bloem, R., Brenguier, R., Könighofer, R., Pérez, G.A., Raskin, J.-F., Ryzhyk, L., Sankur, O., Seidl, M., Tentrup, L., Walker, A.: The second reactive synthesis competition (SYNTCOMP 2015). In: SYNT, EPTCS, vol. 202, pp. 27---57. Open Publishing Association (2016)

[42]

Jacobs, S., Klein, F.: A high-level LTL synthesis format: TLSF v1.0. CoRR (2016). arXiv:1601.05228. Accessed Feb 2016

[43]

Järvisalo, M., Berre, D.L., Roussel, O., Simon, L.: The international SAT solver competitions. AI Mag 33(1), 89---94 (2012)

Digital Library

[44]

Jobstmann, B., Bloem, R.: Optimizations for LTL synthesis. In: FMCAD, pp. 117---124. IEEE Computer Society (2006)

Digital Library

[45]

Jobstmann, B., Galler, S.J., Weiglhofer, M., Bloem, R.: Anzu: A tool for property synthesis. In: CAV, LNCS, vol. 4590, pp. 258---262. Springer (2006)

Digital Library

[46]

Kupferman, O., Vardi, M.Y.: Safraless decision procedures. In: FOCS, pp. 531---542. IEEE Computer Society (2005)

Digital Library

[47]

Kurshan, R.P.: Automata-theoretic verification of coordinating processes. In: Analysis and Optimization of Systems: Discrete Event Systems, pp. 16---28. Springer (1994)

[48]

Lecoutre, C., Roussel, O., van Dongen, M.R.C.: Promoting robust black-box solvers through competitions. Constraints 15(3), 317---326 (2010)

Digital Library

[49]

Mishchenko, A., Chatterjee, S., Brayton, R.K.: Dag-aware AIG rewriting a fresh look at combinational logic synthesis. In: DAC, pp. 532---535. ACM (2006)

Digital Library

[50]

Mishchenko, A., Chatterjee, S., Jiang, R., Brayton, R.: FRAIGs: A unifying representation for logic synthesis and verification. Technical report, EECS Department, U. C. Berkeley (2005)

[51]

Morgenstern, A., Gesell, M., Schneider, K.: Solving games using incremental induction. In: IFM'13, LNCS 7940, pp. 177---191. Springer (2013)

[52]

Niemetz, A., Preiner, M., Lonsing, F., Seidl, M., Biere, A.: Resolution-based certificate extraction for QBF. In: SAT'12, LNCS 7317, pp. 430---435. Springer (2012)

Digital Library

[53]

Pnueli, A., Rosner, R.: On the synthesis of a reactive module. In: POPL, pp. 179---190. ACM Press (1989)

Digital Library

[54]

Rabin, M.O.: Decidability of second-order theories and automata on infinite trees. Trans. Am. Math. Soc. 141, 1---35 (1969)

[55]

Ranjan, R.K., Aziz, A., Brayton, R.K., Plessier, B., Pixley, C.: Efficient bdd algorithms for fsm synthesis and verification. In: International Workshop on Logic Synthesis (1995)

[56]

Roussel, O.: Controlling a solver execution with the runsolver tool. JSAT 7(4), 139---144 (2011)

[57]

Rudell, R.: Dynamic variable ordering for ordered binary decision diagrams. In: ICCAD, pp. 42---47. IEEE Computer Society (1993)

Digital Library

[58]

Seidl, M., Könighofer, R.: Partial witnesses from preprocessed quantified boolean formulas. In: DATE'14, pp. 1---6. IEEE (2014)

Digital Library

[59]

Sohail, S., Somenzi, F.: Safety first: a two-stage algorithm for the synthesis of reactive systems. STTT 15(5---6), 433---454 (2013)

[60]

Somenzi, F.: Binary decision diagrams. In: Calculational System Design, vol. 173, pp. 303. IOS Press (1999)

[61]

Sutcliffe, G., Suttner, C.B.: Evaluating general purpose automated theorem proving systems. Artif. Intell. 131(1---2), 39---54 (2001)

Digital Library

[62]

Sutcliffe, G., Suttner, C.B.: The state of CASC. AI Commun. 19(1), 35---48 (2006)

Digital Library

[63]

Thomas, W.: On the synthesis of strategies in infinite games. In: STACS, pp. 1---13 (1995)

[64]

van Dijk, T., Laarman, A., van de Pol, J.: Multi-core BDD operations for symbolic reachability. Electron. Notes Theor. Comput. Sci. 296, 127---143 (2013)

Digital Library

[65]

van Dijk, T., van de Pol, J.: Sylvan: Multi-core decision diagrams. In: TACAS 2015, LNCS, vol. 9035, pp. 677---691 Springer (2015)

Digital Library

Cited By

Cimatti AGeatti LGigante NMontanari ATonetta S(2024)Extended bounded response LTL: a new safety fragment for efficient reactive synthesisFormal Methods in System Design10.1007/s10703-021-00383-364:1(1-49)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s10703-021-00383-3
Jacobs SPérez GAbraham RBruyère VCadilhac MColange MDelfosse Cvan Dijk TDuret-Lutz AFaymonville PFinkbeiner BKhalimov AKlein FLuttenberger MMeyer KMichaud TPommellet ARenkin FSchlehuber-Caissier PSakr MSickert SStaquet GTamines CTentrup LWalker A(2024)The Reactive Synthesis Competition (SYNTCOMP): 2018–2021International Journal on Software Tools for Technology Transfer (STTT)10.1007/s10009-024-00754-126:5(551-567)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10009-024-00754-1
Artale AGeatti LGigante NMazzullo AMontanari AWilliams BChen YNeville J(2023)Complexity of safety and cosafety fragments of linear temporal logicProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i5.25768(6236-6244)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i5.25768
Show More Cited By

The first reactive synthesis competition (SYNTCOMP 2014)
1. Software and its engineering
  1. Software organization and properties
    1. Software functional properties
      1. Formal methods

Recommendations

Can reactive synthesis and syntax-guided synthesis be friends?
SPLASH Companion 2021: Companion Proceedings of the 2021 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity

While reactive synthesis and syntax-guided synthesis (SyGuS) have seen enormous progress in recent years, combining the two approaches has remained a challenge. In this work, we present the synthesis of reactive programs from Temporal Stream Logic ...
Simple synthesis of reactive systems with tolerance for unexpected environmental behavior
FormaliSE '16: Proceedings of the 4th FME Workshop on Formal Methods in Software Engineering

During the synthesis of reactive systems, if we assume some behavioral property of an environment, any specification must include the environmental constraint in their conditions. In general, the behavior of a synthesized reactive system from such ...
Can reactive synthesis and syntax-guided synthesis be friends?
PLDI 2022: Proceedings of the 43rd ACM SIGPLAN International Conference on Programming Language Design and Implementation

While reactive synthesis and syntax-guided synthesis (SyGuS) have seen enormous progress in recent years, combining the two approaches has remained a challenge. In this work, we present the synthesis of reactive programs from Temporal Stream Logic ...

Comments

Information & Contributors

Information

Published In

cover image International Journal on Software Tools for Technology Transfer (STTT)

International Journal on Software Tools for Technology Transfer (STTT) Volume 19, Issue 3

June 2017

117 pages

ISSN:1433-2779

EISSN:1433-2787

Issue’s Table of Contents

Copyright © Copyright © 2017 Springer-Verlag Berlin Heidelberg.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 01 June 2017

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cimatti AGeatti LGigante NMontanari ATonetta S(2024)Extended bounded response LTL: a new safety fragment for efficient reactive synthesisFormal Methods in System Design10.1007/s10703-021-00383-364:1(1-49)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s10703-021-00383-3
Jacobs SPérez GAbraham RBruyère VCadilhac MColange MDelfosse Cvan Dijk TDuret-Lutz AFaymonville PFinkbeiner BKhalimov AKlein FLuttenberger MMeyer KMichaud TPommellet ARenkin FSchlehuber-Caissier PSakr MSickert SStaquet GTamines CTentrup LWalker A(2024)The Reactive Synthesis Competition (SYNTCOMP): 2018–2021International Journal on Software Tools for Technology Transfer (STTT)10.1007/s10009-024-00754-126:5(551-567)Online publication date: 1-Oct-2024
https://dl.acm.org/doi/10.1007/s10009-024-00754-1
Artale AGeatti LGigante NMazzullo AMontanari AWilliams BChen YNeville J(2023)Complexity of safety and cosafety fragments of linear temporal logicProceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence10.1609/aaai.v37i5.25768(6236-6244)Online publication date: 7-Feb-2023
https://dl.acm.org/doi/10.1609/aaai.v37i5.25768
Renkin FSchlehuber-Caissier PDuret-Lutz APommellet A(2022)Dissecting ltlsyntFormal Methods in System Design10.1007/s10703-022-00407-661:2-3(248-289)Online publication date: 1-Dec-2022
https://dl.acm.org/doi/10.1007/s10703-022-00407-6
Cimatti AGeatti LGigante NMontanari ATonetta S(2021)Fairness, Assumptions, and Guarantees for Extended Bounded Response LTL+P SynthesisSoftware Engineering and Formal Methods10.1007/978-3-030-92124-8_20(351-371)Online publication date: 6-Dec-2021
https://dl.acm.org/doi/10.1007/978-3-030-92124-8_20
Mavridou AKatis AGiannakopoulou DKooi DPressburger TWhalen M(2021)From Partial to Global Assume-Guarantee Contracts: Compositional Realizability Analysis in FRETFormal Methods10.1007/978-3-030-90870-6_27(503-523)Online publication date: 20-Nov-2021
https://dl.acm.org/doi/10.1007/978-3-030-90870-6_27
Jacobs SSakr M(2021)AIGEN: Random Generation of Symbolic Transition SystemsComputer Aided Verification10.1007/978-3-030-81688-9_20(435-446)Online publication date: 20-Jul-2021
https://dl.acm.org/doi/10.1007/978-3-030-81688-9_20
Dartois LFiliot ELhote N(2018)Logics for Word Transductions with SynthesisProceedings of the 33rd Annual ACM/IEEE Symposium on Logic in Computer Science10.1145/3209108.3209181(295-304)Online publication date: 9-Jul-2018
https://dl.acm.org/doi/10.1145/3209108.3209181

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents