skip to main content
10.1145/3336294.3336297acmotherconferencesArticle/Chapter ViewAbstractPublication PagessplcConference Proceedingsconference-collections
research-article

Uniform Random Sampling Product Configurations of Feature Models That Have Numerical Features

Published: 09 September 2019 Publication History

Abstract

Analyses of Software Product Lines (SPLs) rely on automated solvers to navigate complex dependencies among features and find legal configurations. Often these analyses do not support numerical features with constraints because propositional formulas use only Boolean variables. Some automated solvers can represent numerical features natively, but are limited in their ability to count and Uniform Random Sample (URS) configurations, which are key operations to derive unbiased statistics on configuration spaces.
Bit-blasting is a technique to encode numerical constraints as propositional formulas. We use bit-blasting to encode Boolean and numerical constraints so that we can exploit existing #SAT solvers to count and URS configurations. Compared to state-of-art Satisfiability Modulo Theory and Constraint Programming solvers, our approach has two advantages: 1) faster and more scalable configuration counting and 2) reliable URS of SPL configurations. We also show that our work can be used to extend prior SAT-based SPL analyses to support numerical features and constraints.

References

[1]
Mustafa Al-Hajjaji, Sebastian Krieter, Thomas Thüm, Malte Lochau, and Gunter Saake. 2016. IncLing: efficient product-line testing using incremental pairwise sampling. In ACM SIGPLAN Notices, Vol. 52. ACM, ACM, ACM, 144--155.
[2]
Sven Apel, Don Batory, Christian Kástner, and Gunter Saake. 2016. Feature-oriented software product lines. Springer, NY, USA.
[3]
Clark Barrett. 2013. Decision Procedures: An Algorithmic Point of View, by Daniel Kroening and Ofer Strichman, Springer-Verlag, 2008. Journal of Automated Reasoning 51, 4 (2013), 453--456.
[4]
Clark Barrett, Christopher L Conway, Morgan Deters, Liana Hadarean, Dejan Jovanović, Tim King, Andrew Reynolds, and Cesare Tinelli. 2011. Cvc4. In International Conference on Computer Aided Verification. Springer, Springer, NY, USA, 171--177.
[5]
Clark Barrett, Robert Nieuwenhuis, Albert Oliveras, and Cesare Tinelli. 2006. Splitting on demand in SAT modulo theories. In International Conference on Logic for Programming Artificial Intelligence and Reasoning. Springer, Springer, NY, USA, 512--526.
[6]
Clark Barrett and Cesare Tinelli. 2018. Satisfiability modulo theories. In Handbook of Model Checking. Springer, NY, USA, 305--343.
[7]
Don Batory. 2005. Feature models, grammars, and propositional formulas. In International Conference on Software Product Lines. Springer, Springer, NY, USA, 7--20.
[8]
David Benavides, Sergio Segura, and Antonio Ruiz-Cortés. 2010. Automated analysis of feature models 20 years later: A literature review. Information Systems 35, 6 (2010), 615--636.
[9]
David Benavides, Sergio Segura, Pablo Trinidad, and Antonio Ruiz-Cortés. 2005. Using Java CSP solvers in the automated analyses of feature models. In International Summer School on Generative and Transformational Techniques in Software Engineering. Springer, Springer, NY, USA, 399--408.
[10]
David Benavides, Sergio Segura, Pablo Trinidad, and Antonio Ruiz-Cortés. 2006. A first step towards a framework for the automated analysis of feature models. Proc. Managing Variability for Software Product Lines: Working With Variability Mechanisms 0 (2006), 39--47.
[11]
Thorsten Berger, Steven She, Rafael Lotufo, Andrzej Wasowski, and Krzysztof Czarnecki. 2013. A study of variability models and languages in the systems software domain. IEEE Transactions on Software Engineering 39, 12 (2013), 1611--1640.
[12]
Armin Biere, Marijn Heule, and Hans van Maaren. 2009. Handbook of satisfiability. Vol. 185. IOS press, IEEE.
[13]
Angelo Brillout, Daniel Kroening, and Thomas Wahl. 2009. Mixed abstractions for floating-point arithmetic. In 2009 Formal Methods in Computer-Aided Design. IEEE, IEEE, Piscataway, NJ, USA, 69--76.
[14]
Robert Brummayer and Armin Biere. 2009. Boolector: An efficient SMT solver for bit-vectors and arrays. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, Springer, NY, USA, 174--177.
[15]
Randal E Bryant, Daniel Kroening, Joël Ouaknine, Sanjit A Seshia, Ofer Strichman, and Bryan Brady. 2007. Deciding bit-vector arithmetic with abstraction. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, Springer, NY, USA, 358--372.
[16]
Gianpiero Cabodi, Paolo Camurati, Luciano Lavagno, and Stefano Quer. 1997. Disjunctive partitioning and partial iterative squaring: an effective approach for symbolic traversal of large circuits. In Proceedings of the 34th annual Design Automation Conference. ACM, ACM, New York, NY, USA, 728--733.
[17]
Supratik Chakraborty, Daniel J. Fremont, Kuldeep S. Meel, Sanjit A. Seshia, and Moshe Y. Vardi. 2015. On Parallel Scalable Uniform SAT Witness Generation. In Tools and Algorithms for the Construction and Analysis of Systems, Christel Baier and Cesare Tinelli (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 304--319.
[18]
Jianfeng Chen, Vivek Nair, Rahul Krishna, and Tim Menzies. 2018. " Sampling" as a Baseline Optimizer for Search-based Software Engineering. IEEE Transactions on Software Engineering 0 (2018), 12.
[19]
Maciej Ciesielski, Walter Brown, and André Rossi. 2013. Arithmetic bit-level verification using network flow model. In Haifa Verification Conference. Springer, Springer, NY, USA, 327--343.
[20]
Alessandro Cimatti, Alberto Griggio, Bastiaan Joost Schaafsma, and Roberto Sebastiani. 2013. The mathsat5 smt solver. In International Conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, Springer, Springer, 93--107.
[21]
Maxime Cordy, Pierre-Yves Schobbens, Patrick Heymans, and Axel Legay. 2013. Beyond boolean product-line model checking: dealing with feature attributes and multi-features. In Proceedings of the 2013 International Conference on Software Engineering. IEEE Press, IEEE, IEEE, 472--481.
[22]
Leonardo De Moura and Nikolaj Bjørner. 2008. Z3: An efficient SMT solver. In International conference on Tools and Algorithms for the Construction and Analysis of Systems. Springer, Springer, NY, USA, 337--340.
[23]
Bruno Dutertre. 2014. Yices 2.2. In International Conference on Computer Aided Verification. Springer, Springer, NY, USA, 737--744.
[24]
Rafael Dutra, Kevin Laeufer, Jonathan Bachrach, and Koushik Sen. 2018. Efficient sampling of SAT solutions for testing. In 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE). IEEE, ACM, New York, NY, USA, 549--559.
[25]
Holger Eichelberger and Klaus Schmid. 2013. A systematic analysis of textual variability modeling languages. In Proceedings of the 17th International Software Product Line Conference. ACM, ACM, New York, NY, USA, 12--21.
[26]
Sascha El-Sharkawy, Adam Krafczyk, and Klaus Schmid. 2017. An empirical study of configuration mismatches in linux. In Proceedings of the 21st International Systems and Software Product Line Conference-Volume A. ACM, ACM, New York, NY, USA, 19--28.
[27]
Moshe Emmer, Zurab Khasidashvili, Konstantin Korovin, and Andrei Voronkov. 2010. Encoding industrial hardware verification problems into effectively propositional logic. In Proceedings of the 2010 Conference on Formal Methods in Computer-Aided Design. FMCAD Inc, Springer, NY, USA, 137--144.
[28]
The Linux Foundation. 2018. Kconfig-language specification. https://www.kernel.org/doc/Documentation/kbuild/kconfig-language.txt.
[29]
The Linux Foundation. 2018. Kconfig tool specification. https://www.kernel.org/doc/Documentation/kbuild/kconfig.txt.
[30]
Vijay Ganesh and David L Dill. 2006. System description of STP.
[31]
Paul Gazzillo. 2017. Kmax: finding all configurations of Kbuild makefiles statically. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, ACM, ACM, 279--290.
[32]
Alexander Grebhahn, Sebastian Kuckuk, Christian Schmitt, Harald Köstler, Norbert Siegmund, Sven Apel, Frank Hannig, and Jürgen Teich. 2014. Experiments on optimizing the performance of stencil codes with spl conqueror. Parallel Processing Letters 24, 03 (2014), 1441001.
[33]
Jianmei Guo, Krzysztof Czarnecki, Sven Apel, Norbert Siegmund, and Andrzej Wasowski. 2013. Variability-aware Performance Prediction: A Statistical Learning Approach. In Proceedings of the 28th IEEE/ACM International Conference on Automated Software Engineering (ASE'13). IEEE Press, Piscataway, NJ, USA, 301--311.
[34]
Jianmei Guo, Jia Hui Liang, Kai Shi, Dingyu Yang, Jingsong Zhang, Krzysztof Czarnecki, Vijay Ganesh, and Huiqun Yu. 2017. SMTIBEA: a hybrid multi-objective optimization algorithm for configuring large constrained software product lines. Software & Systems Modeling 0 (2017), 1--20.
[35]
Christopher Henard, Mike Papadakis, Mark Harman, and Yves Le Traon. 2015. Combining multi-objective search and constraint solving for configuring large software product lines. In Proceedings of the 37th International Conference on Software Engineering-Volume 1. IEEE Press, IEEE/ACM, Piscataway, NJ, USA, 517--528.
[36]
Christopher Henard, Mike Papadakis, Mark Harman, and Yves Le Traon. 2015. Combining Multi-objective Search and Constraint Solving for Configuring Large Software Product Lines. In Proceedings of the 37th International Conference on Software Engineering - Volume 1 (ICSE '15). IEEE Press, Piscataway, NJ, USA, 517--528. http://dl.acm.org/citation.cfm?id=2818754.2818819
[37]
Jose-Miguel Horcas, Mónica Pinto, and Lidia Fuentes. 2018. Variability models for generating efficient configurations of functional quality attributes. Information and Software Technology 95 (2018), 147--164.
[38]
Jeho Oh, Paul Gazzillo, Don Batory, Marijn Heule, and Maggie Myers. 2019. Uniform Sampling from Kconfig Feature Models. Technical Report TR-19-02. University of Texas at Austin, Department of Computer Science.
[39]
Martin Jonas. 2016. SMT Solving for the Theory of Bit-Vectors. Ph.D. Dissertation. Masaryk University.
[40]
Ahmet Serkan Karataş, Halit Oğuztüzün, and Ali Doğru. 2013. From extended feature models to constraint logic programming. Science of Computer Programming 78, 12 (2013), 2295--2312.
[41]
Christian Kästner, Paolo G Giarrusso, Tillmann Rendel, Sebastian Erdweg, Klaus Ostermann, and Thorsten Berger. 2011. Variability-aware parsing in the presence of lexical macros and conditional compilation. In ACM SIGPLAN Notices, Vol. 46. ACM, IEEE/ACM, Piscataway, NJ, USA, 805--824.
[42]
Jia Hui Liang, Vijay Ganesh, Krzysztof Czarnecki, and Venkatesh Raman. 2015. SAT-based analysis of large real-world feature models is easy. In Proceedings of the 19th International Conference on Software Product Line. ACM, IEEE/ACM, Piscataway, NJ, USA, 91--100.
[43]
Jörg Liebig, Alexander Von Rhein, Christian Kästner, Sven Apel, Jens Dörre, and Christian Lengauer. 2013. Scalable analysis of variable software. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. ACM, ACM, New York, NY, USA, 81--91.
[44]
Henry B Mann and Donald R Whitney. 1947. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics 0, 0 (1947), 50--60.
[45]
Frank J Massey Jr. 1951. The Kolmogorov-Smirnov test for goodness of fit. Journal of the American statistical Association 46, 253 (1951), 68--78.
[46]
Jacopo Mauro, Michael Nieke, Christoph Seidl, and Ingrid Chieh Yu. 2017. Anomaly Detection and Explanation in Context-Aware Software Product Lines. In Proceedings of the 21st International Systems and Software Product Line Conference-Volume B. ACM, IEEE/ACM, Piscataway, NJ, USA, 18--21.
[47]
Flávio Medeiros, Christian Kästner, Márcio Ribeiro, Rohit Gheyi, and Sven Apel. 2016. A comparison of 10 sampling algorithms for configurable systems. In Proceedings of the 38th International Conference on Software Engineering. ACM, IEEE/ACM, Piscataway, NJ, USA, 643--654.
[48]
Daniel-Jesus Munoz, José A. Montenegro, Mónica Pinto, and Lidia Fuentes. 2019. Energy-aware environments for the development of green applications for cyber-physical systems. Future Generation Computer Systems 91 (2019), 536 -- 554.
[49]
Daniel-Jesus Munoz, Mónica Pinto, and Lidia Fuentes. 2018. Finding correlations of features affecting energy consumption and performance of web servers using theHADAS eco-assistant. Computing 100, 11 (01 Nov 2018), 1155--1173.
[50]
Sarah Nadi and Ric Holt. 2012. Mining Kbuild to detect variability anomalies in Linux. In Software Maintenance and Reengineering (CSMR), 2012 16th European Conference on. IEEE, IEEE/ACM, Piscataway, NJ, USA, 107--116.
[51]
Jeho Oh, Don Batory, Margaret Myers, and Norbert Siegmund. 2017. Finding near-optimal configurations in product lines by random sampling. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering. ACM, IEEE/ACM, Piscataway, NJ, USA, 61--71.
[52]
Quentin Plazar, Mathieu Acher, Gilles Perrouin, Xavier Devroey, and Maxime Cordy. 2019. Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet?. In ICST 2019 - 12th International Conference on Software Testing, Verification, and Validation. HAL-Inria, Xian, China, 1--12. https://hal.inria.fr/hal-01991857
[53]
Francesca Rossi, Peter Van Beek, and Toby Walsh. 2006. Handbook of constraint programming. Elsevier, Elsevier.
[54]
Luis Emiliano Sánchez, Jorge Andrés Diaz-Pace, and Alejandro Zunino. 2019. A family of heuristic search algorithms for feature model optimization. Science of Computer Programming 172 (2019), 264 -- 293.
[55]
Atri Sarkar, Jianmei Guo, Norbert Siegmund, Sven Apel, and Krzysztof Czarnecki. 2015. Cost-efficient sampling for performance prediction of configurable systems (t). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, IEEE/ACM, Piscataway, NJ, USA, 342--352.
[56]
A. S. Sayyad, J. Ingram, T. Menzies, and H. Ammar. 2013. Scalable product line configuration: A straw to break the camel's back. In 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE/ACM, ASE, 465--474.
[57]
K. Shi. 2017. Combining Evolutionary Algorithms with Constraint Solving for Configuration Optimization. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME). IEEE/ACM, Piscataway, NJ, USA, 665--669.
[58]
Norbert Siegmund, Alexander Grebhahn, Sven Apel, and Christian Kästner. 2015. Performance-influence Models for Highly Configurable Systems. In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2015). ACM, New York, NY, USA, 284--294.
[59]
Norbert Siegmund, Sergiy S. Kolesnikov, Christian Kästner, Sven Apel, Don Batory, Marko Rosenmüller, and Gunter Saake. 2012. Predicting Performance via Automated Feature-interaction Detection. In Proceedings of the 34th International Conference on Software Engineering (ICSE '12). IEEE Press, Piscataway, NJ, USA, 167--177. http://dl.acm.org/citation.cfm?id=2337223.2337243
[60]
Niklas Sorensson and Niklas Een. 2005. Minisat v1. 13-a sat solver with conflict-clause minimization. SAT 2005, 53 (2005), 1--2.
[61]
Creative Research Systems. 2019. Sample Size Calculator. https://www.surveysystem.com/sscalc.htm.
[62]
Reinhard Tartler, Daniel Lohmann, Christian Dietrich, Christoph Egger, and Julio Sincero. 2011. Configuration Coverage in the Analysis of Large-scale System Software. In Proceedings of the 6th Workshop on Programming Languages and Operating Systems (PLOS '11). ACM, New York, NY, USA, Article 2, 5 pages.
[63]
Thomas Thüm, Christian Kästner, Fabian Benduhn, Jens Meinicke, Gunter Saake, and Thomas Leich. 2014. FeatureIDE: An extensible framework for feature-oriented software development. Science of Computer Programming 79 (2014), 70--85.
[64]
Marc Thurley. 2006. sharpSAT-counting models with advanced component caching and implicit BCP. In International Conference on Theory and Applications of Satisfiability Testing. Springer, Springer Berlin Heidelberg, Berlin, Heidelberg, 424--429.
[65]
G. S. Tseitin. 1983. On the Complexity of Derivation in Propositional Calculus. Springer Berlin Heidelberg, Berlin, Heidelberg, 466--483.
[66]
Christoph M Wintersteiger, Youssef Hamadi, and Leonardo De Moura. 2013. Efficiently solving quantified bit-vector formulas. Formal Methods in System Design 42, 1 (2013), 3--23.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
SPLC '19: Proceedings of the 23rd International Systems and Software Product Line Conference - Volume A
September 2019
356 pages
ISBN:9781450371384
DOI:10.1145/3336294
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Permissions

Request permissions for this article.

Check for updates

Badges

Author Tags

  1. bit-blasting
  2. feature model
  3. model counting
  4. numerical features
  5. propositional formula
  6. software product lines

Qualifiers

  • Research-article

Funding Sources

  • Center for Hierarchical Manufacturing, National Science Foundation
  • Consejería de Economía, Innovación, Ciencia y Empleo, Junta de Andalucía
  • Ministerio de Economía y Competitividad

Conference

SPLC 2019

Acceptance Rates

Overall Acceptance Rate 167 of 463 submissions, 36%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)43
  • Downloads (Last 6 weeks)9
Reflects downloads up to 05 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media