Article

Predicting performance via automated feature-interaction detection

Authors:

Norbert Siegmund,

Sergiy S. Kolesnikov,

Christian Kästner,

Marko Rosenmüller,

Gunter SaakeAuthors Info & Claims

ICSE '12: Proceedings of the 34th International Conference on Software Engineering

Pages 167 - 177

Published: 02 June 2012 Publication History

Abstract

Customizable programs and program families provide user-selectable features to allow users to tailor a program to an application scenario. Knowing in advance which feature selection yields the best performance is difficult because a direct measurement of all possible feature combinations is infeasible. Our work aims at predicting program performance based on selected features. However, when features interact, accurate predictions are challenging. An interaction occurs when a particular feature combination has an unexpected influence on performance. We present a method that automatically detects performance-relevant feature interactions to improve prediction accuracy. To this end, we propose three heuristics to reduce the number of measurements required to detect interactions. Our evaluation consists of six real-world case studies from varying domains (e.g., databases, encoding libraries, and web servers) using different configuration techniques (e.g., configuration files and preprocessor flags). Results show an average prediction accuracy of 95%.

References

[1]

A. Rabkin and R. Katz, "Static extraction of program configuration options," in ICSE. ACM, 2011, pp. 131-140.

Digital Library

[2]

K. Czarnecki and U. Eisenecker, Generative Programming: Methods, Tools, and Applications. Addison-Wesley, 2000.

Digital Library

[3]

N. Siegmund, M. Rosenmüller, C. Kästner, P. Giarrusso, S. Apel, and S. Kolesnikov, "Scalable prediction of nonfunctional properties in software product lines," in SPLC. IEEE, 2011, pp. 160-169.

Digital Library

[4]

J. Sincero, W. Schroder-Preikschat, and O. Spinczyk, "Approaching non-functional properties of software product lines: Learning from products," in APSEC. IEEE, 2010, pp. 147- 155.

Digital Library

[5]

M. Calder, M. Kolberg, E. Magill, and S. Reiff-Marganiec, "Feature interaction: A critical review and considered forecast," Comput. Netw., vol. 41, no. 1, pp. 115-141, 2003.

Digital Library

[6]

A. Nhlabatsi, R. Laney, and B. Nuseibeh, "Feature interaction: The security threat from within software systems," Progress in Informatics, no. 5, pp. 75-89, 2008.

[7]

M. Calder and A. Miller, "Feature interaction detection by pairwise analysis of LTL properties: A case study," Formal Methods System Design, vol. 28, no. 3, pp. 213-261, 2006.

Digital Library

[8]

N. Siegmund, M. Rosenmüller, M. Kuhlemann, C. Kästner, S. Apel, and G. Saake, "SPL Conqueror: Toward optimization of non-functional properties in software product lines," Software Quality Journal, 2011, online first.

Digital Library

[9]

D. Batory, P. Höfner, and J. Kim, "Feature interactions, products, and composition," in GPCE. ACM, 2011, pp. 13-22.

Digital Library

[10]

J. Lee, K. Kang, and S. Kim, "A feature-based approach to product line production planning," in Software Product Lines, ser. LNCS. Springer, 2004, vol. 3154, pp. 137-140.

[11]

C. Kim, C. Kästner, and D. Batory, "On the modularity of feature interactions," in GPCE. ACM, 2008, pp. 23-34.

Digital Library

[12]

D. Cohen, S. Dalal, J. Parelius, and G. Patton, "The combinatorial design approach to automatic test generation," IEEE Software, vol. 13, no. 5, pp. 83-88, 1996.

Digital Library

[13]

K.-C. Tai and Y. Lei, "A test generation strategy for pairwise testing," IEEE TSE, vol. 28, no. 1, pp. 109-111, 2002.

Digital Library

[14]

A. Williams, "Determination of test configurations for pairwise interaction coverage," in TestComm. Kluwer, 2000, pp. 59-74.

Digital Library

[15]

S. Oster, F. Markert, and P. Ritter, "Automated incremental pairwise testing of software product lines," in SPLC, 2010, pp. 196-210.

Digital Library

[16]

J. Liebig, S. Apel, C. Lengauer, C. Kästner, and M. Schulze, "An analysis of the variability in forty preprocessor-based software product lines," in ICSE. ACM, 2010, pp. 105-114.

Digital Library

[17]

S. Apel and D. Beyer, "Feature cohesion in software product lines: An exploratory study," in ICSE. ACM, 2011, pp. 421-430.

Digital Library

[18]

C. Taube-Schock, R. Walker, and I. Witten, "Can we avoid high coupling?" in ECOOP. Springer, 2011, pp. 204-228.

Digital Library

[19]

S. She, R. Lotufo, T. Berger, A. Wasowski, and K. Czarnecki, "Reverse engineering feature models," in ICSE. ACM, 2011, pp. 461-470.

Digital Library

[20]

T. Mytkowicz, A. Diwan, M. Hauswirth, and P. Sweeney, "Producing wrong data without doing anything obviously wrong!" in ASPLOS. ACM, 2009, pp. 265-276.

Digital Library

[21]

A. Georges, D. Buytaert, and L. Eeckhout, "Statistically rigorous Java performance evaluation," in OOPSLA. ACM, 2007, pp. 57-76.

Digital Library

[22]

A. Abdelaziz, W. Kadir, and A. Osman, "Comparative analysis of software performance prediction approaches in context of component-based system," IJCA, vol. 23, no. 3, pp. 15-22, 2011.

[23]

S. Balsamo, A. Di Marco, P. Inverardi, and M. Simeoni, "Model-based performance prediction in software development: A survey," IEEE TSE, vol. 30, no. 5, pp. 295-310, 2004.

Digital Library

[24]

I. Witten and E. Frank, Data mining : Practical machine learning tools and techniques, 2nd ed. Elsevier, Morgan Kaufman, 2005.

Digital Library

[25]

F. Jensen and T. Nielsen, Bayesian Networks and Decision Graphs, 2nd ed. Springer, 2007.

Digital Library

[26]

K. Mardia, J. Kent, and J. Bibby, Multivariate Analysis (Probability and Mathematical Statistics), 1st ed. Academic Press, 1980.

[27]

H. Hotelling, "Analysis of a complex of statistical variables into principal components," Journal of Educational Psychology, vol. 24, no. 6, pp. 417-441, 1933.

[28]

A. Ganapathi, H. Kuno, U. Dayal, J. Wiener, A. Fox, M. Jordan, and D. Patterson, "Predicting multiple metrics for queries: Better decisions enabled by machine learning," in ICDE. IEEE, 2009, pp. 592-603.

Digital Library

[29]

K. Krogmann, M. Kuperberg, and R. Reussner, "Using genetic search for reverse engineering of parametric behavior models for performance prediction," IEEE TSE, vol. 36, no. 6, pp. 865-877, 2010.

Digital Library

[30]

J. Happe, H. Koziolek, and R. Reussner, "Facilitating performance predictions using software components," IEEE Software, vol. 28, no. 3, pp. 27-33, 2011.

Digital Library

[31]

R. Tawhid and D. Petriu, "Automatic derivation of a product performance model from a software product line model," in SPLC. IEEE, 2011, pp. 80-89.

Digital Library

[32]

S. Chen, Y. Liu, I. Gorton, and A. Liu, "Performance prediction of component-based applications," Journal of System Software, vol. 74, no. 1, pp. 35-43, 2005.

Digital Library

[33]

S. Yacoub, "Performance analysis of component-based applications," in SPLC. Springer, 2002, vol. 2379, pp. 1-5.

Digital Library

[34]

M. Calder, M. Kolberg, E. Magill, and S. Reiff-Marganiec, "Feature interaction: A critical review and considered forecast," Comp. Netw. and ISDN Systems, vol. 41, pp. 115-141, 2003.

Digital Library

[35]

C. Prehofer, "Plug-and-play composition of features and feature interactions with statechart diagrams," Software and Systems Modeling, vol. 3, no. 3, pp. 221-234, 2004.

Digital Library

[36]

K. Pomakis and J. Atlee, "Reachability analysis of feature interactions: A progress report," in International Symposium on Software Testing and Analysis, vol. 21. ACM, 1996, pp. 216-223.

Digital Library

[37]

A. Classen, P. Heymans, P.-Y. Schobbens, A. Legay, and J.-F. Raskin, "Model checking lots of systems: Efficient verification of temporal properties in software product lines," in ICSE. ACM, 2010, pp. 335-344.

Digital Library

[38]

K. Lauenroth, K. Pohl, and S. Toehning, "Model checking of domain artifacts in product line engineering," in ASE. IEEE, 2009, pp. 269-280.

Digital Library

[39]

S. Apel, W. Scholz, C. Lengauer, and C. Kästner, "Detecting dependences and interactions in feature-oriented design," in ISSRE. IEEE, 2010, pp. 161-170.

Digital Library

[40]

S. Apel, H. Speidel, P. Wendler, A. von Rhein, and D. Beyer, "Detection of feature interactions using feature-aware verification," in ASE. IEEE, 2011, pp. 372-375.

Digital Library

[41]

J. Liu, D. Batory, and S. Nedunuri, "Modeling interactions in feature-oriented designs," in ICFI. IOS Press, 2005, pp. 178-197.

Cited By

Mühlbauer SSattler FKaltenecker CDorn JApel SSiegmund NGrundy JPollock LPenta M(2023)Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software SystemsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00176(2085-2097)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00176
Chen TSchmerl BMaggio MCámara J(2022)Planning landscape analysis for self-adaptive systemsProceedings of the 17th Symposium on Software Engineering for Adaptive and Self-Managing Systems10.1145/3524844.3528060(84-90)Online publication date: 18-May-2022
https://dl.acm.org/doi/10.1145/3524844.3528060
Dubslaff CWeis KBaier CApel SDwyer MDamian DZeller A(2022)Causality in configurable software systemsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510200(325-337)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510200
Show More Cited By

Recommendations

Predicting Group Performance in Task-Based Interaction
ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal Interaction

We address the problem of automatically predicting group performance on a task, using multimodal features derived from the group conversation. These include acoustic features extracted from the speech signal, and linguistic features derived from the ...
Predicting query performance using query, result, and user interaction features
RIAO '10: Adaptivity, Personalization and Fusion of Heterogeneous Information

The high cost of search engine evaluation makes techniques for accurately predicting engine effectiveness valuable. In this paper we present a study in which we use features of the query, search results, and user interaction with the search results to ...
Feature-based deep neural network approach for predicting mortality risk in patients with COVID-19
Abstract
In this study, we integrate deep neural network (DNN) with hybrid approaches (feature selection and instance clustering) to build prediction models for predicting mortality risk in patients with COVID-19. Besides, we use cross-validation methods ...
Highlights
- We integrate feature selection, instance clustering, and DNN techniques to improve prediction performance.
- The proposed approach which uses fewer features to build a DNN prediction model with high prediction performance as well as the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '12: Proceedings of the 34th International Conference on Software Engineering

June 2012

1657 pages

ISBN:9781467310673

General Chair:
Martin Glinz,
Program Chairs:
Gail Murphy,
Mauro Pezzè

Sponsors

SIGSOFT: ACM Special Interest Group on Software Engineering

Publisher

IEEE Press

Publication History

Published: 02 June 2012

Check for updates

Qualifiers

Article

Conference

ICSE '12

Sponsor:

SIGSOFT

ICSE '12: 34th International Conference on Software Engineering

June 2 - 9, 2012

Zurich, Switzerland

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

79
Total Citations
View Citations
453
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Mühlbauer SSattler FKaltenecker CDorn JApel SSiegmund NGrundy JPollock LPenta M(2023)Analyzing the Impact of Workloads on Modeling the Performance of Configurable Software SystemsProceedings of the 45th International Conference on Software Engineering10.1109/ICSE48619.2023.00176(2085-2097)Online publication date: 14-May-2023
https://dl.acm.org/doi/10.1109/ICSE48619.2023.00176
Chen TSchmerl BMaggio MCámara J(2022)Planning landscape analysis for self-adaptive systemsProceedings of the 17th Symposium on Software Engineering for Adaptive and Self-Managing Systems10.1145/3524844.3528060(84-90)Online publication date: 18-May-2022
https://dl.acm.org/doi/10.1145/3524844.3528060
Dubslaff CWeis KBaier CApel SDwyer MDamian DZeller A(2022)Causality in configurable software systemsProceedings of the 44th International Conference on Software Engineering10.1145/3510003.3510200(325-337)Online publication date: 21-May-2022
https://dl.acm.org/doi/10.1145/3510003.3510200
Guégain ÉQuinton CRouvoy RMousavi MSchobbens P(2021)On reducing the energy consumption of software product linesProceedings of the 25th ACM International Systems and Software Product Line Conference - Volume A10.1145/3461001.3471142(89-99)Online publication date: 6-Sep-2021
https://dl.acm.org/doi/10.1145/3461001.3471142
Han XYu TPradel MBourcier JJiang ZBezemer CCortellessa V(2021)ConfProf: White-Box Performance Profiling of Configuration OptionsProceedings of the ACM/SPEC International Conference on Performance Engineering10.1145/3427921.3450255(1-8)Online publication date: 9-Apr-2021
https://dl.acm.org/doi/10.1145/3427921.3450255
Chen JXu NChen PZhang H(2021)Efficient Compiler Autotuning via Bayesian OptimizationProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00110(1198-1209)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00110
Velez MJamshidi PSiegmund NApel SKästner C(2021)White-Box Analysis over Machine LearningProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00100(1072-1084)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00100
Weber MApel SSiegmund N(2021)White-Box Performance-Influence ModelsProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00099(1059-1071)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00099
Nusrat FHassan FZhong HWang X(2021)How Developers Optimize Virtual Reality ApplicationsProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00052(473-485)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00052
Gao YZhu YZhang HLin HYang M(2021)Resource-Guided Configuration Space Reduction for Deep Learning ModelsProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00028(175-187)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00028
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents