research-article

Bow tie testing: a testing pattern for product lines

Authors:

Martin F. Johansen,

Øystein Haugen,

Franck FleureyAuthors Info & Claims

EuroPLoP '11: Proceedings of the 16th European Conference on Pattern Languages of Programs

Article No.: 9, Pages 1 - 13

https://doi.org/10.1145/2396716.2396725

Published: 13 July 2011 Publication History

Get Access

Abstract

Verification of highly configurable systems poses a significant challenge, the challenge of knowing that every configuration works when there often are intractably many different configurations. When a homogeneous abstraction layer has many mutually exclusive alternative implementations, we might, according to the polymorphic server test pattern, test these implementations using one test suite targeted towards the abstraction layer which is then run for each concrete implementation of the abstraction layer. But, the pattern does not handle interaction testing. Combinatorial interaction testing is one of the more promising techniques for doing interaction testing of a software product line. The bow tie testing pattern describes how the configurations which differ only in the implementation layer require one test suite. In addition, comparing the execution results of one product with another provides for a test oracle. The pattern reduces the effort of testing a highly configurable system without reducing the error detection capabilities provided by ordinary combinatorial interaction testing. We present an example of a subset of the Eclipse IDE product line, and show that only 20 test suites is required to test 41 products, a significant reduction.

References

[1]

Binder, R. V. 1999. Testing object-oriented systems: models, patterns, and tools. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.

Digital Library

Google Scholar

[2]

Bosch, J. and Lee, J., Eds. 2010. Software Product Lines: Going Beyond - 14th International Conference, SPLC 2010, Jeju Island, South Korea, September 13--17, 2010. Proceedings. Lecture Notes in Computer Science Series, vol. 6287. Springer.

Digital Library

Google Scholar

[3]

Cabral, I., Cohen, M. B., and Rothermel, G. 2010. Improving the testing and testability of software product lines. See Bosch and Lee {2010}, 241--255.

Digital Library

Google Scholar

[4]

Cohen, M. B., Dwyer, M. B., and Shi, J. 2008. Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. IEEE Transactions on Software Engineering 34, 633--650.

Digital Library

Google Scholar

[5]

Engström, E. and Runeson, P. 2011. Software product line testing - a systematic mapping study. Information and Software Technology 53, 1, 2--13.

Digital Library

Google Scholar

[6]

Johansen, M. F., Haugen, Ø., and Fleurey, F. 2011. A Survey of Empirics of Strategies for Software Product Line Testing. In Proceedings of the 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, ICSTW '11, L. O'Conner, Ed. IEEE Computer Society, Washington, DC, USA, 266--269.

Digital Library

Google Scholar

[7]

Kang, K. C., Cohen, S. G., Hess, J. A., Novak, W. E., and Peterson, A. S. 1990. Feature-oriented domain analysis (foda) feasibility study. Tech. rep., Carnegie-Mellon University Software Engineering Institute. November.

Google Scholar

[8]

Kuhn, D. R., Wallace, D. R., and Gallo, A. M. 2004. Software fault interactions and implications for software testing. IEEE Transactions on Software Engineering 30, 6, 418--421.

Digital Library

Google Scholar

[9]

Pohl, K., Böckle, G., and Linden, F. J. v. d. 2005. Software Product Line Engineering: Foundations, Principles and Techniques. Springer-Verlag New York, Inc., Secaucus, NJ, USA.

Digital Library

Google Scholar

[10]

Reuys, A., Reis, S., Kamsties, E., and Pohl, K. 2006. The scented method for testing software product lines. In Software Product Lines, T. Käkölä and J. C. Dueñas, Eds. Springer, Berlin, Heidelberg, 479--520.

Google Scholar

[11]

Rivieres, J. and Beaton, W. 2006. Eclipse Platform Technical Overview.

Google Scholar

[12]

Stricker, V., Metzger, A., and Pohl, K. 2010. Avoiding redundant testing in application engineering. See Bosch and Lee {2010}, 226--240.

Digital Library

Google Scholar

[13]

Uzuncaova, E., Khurshid, S., and Batory, D. 2010. Incremental test generation for software product lines. Software Engineering, IEEE Transactions on 36, 3, 309--322.

Digital Library

Google Scholar

Cited By

View all

Johansen MHaugen ØFleurey F(2011)Properties of realistic feature models make combinatorial testing of product lines feasibleProceedings of the 14th international conference on Model driven engineering languages and systems10.5555/2050655.2050721(638-652)Online publication date: 16-Oct-2011
https://dl.acm.org/doi/10.5555/2050655.2050721
Johansen MHaugen ØFleurey F(2011)Properties of Realistic Feature Models Make Combinatorial Testing of Product Lines FeasibleModel Driven Engineering Languages and Systems10.1007/978-3-642-24485-8_47(638-652)Online publication date: 2011
https://doi.org/10.1007/978-3-642-24485-8_47

Index Terms

Bow tie testing: a testing pattern for product lines
1. Software and its engineering

Recommendations

Model-based pairwise testing for feature interaction coverage in software product line engineering

Testing software product lines (SPLs) is very challenging due to a high degree of variability leading to an enormous number of possible products. The vast majority of today's testing approaches for SPLs validate products individually using different ...
Pairwise testing for software product lines: comparison of two approaches

Software Product Lines (SPL) are difficult to validate due to combinatorics induced by variability, which in turn leads to combinatorial explosion of the number of derivable products. Exhaustive testing in such a large products space is hardly feasible. ...
Multi-objective test prioritization in software product line testing: an industrial case study
SPLC '14: Proceedings of the 18th International Software Product Line Conference - Volume 1

Test prioritization is crucial for testing products in a product line considering limited budget in terms of available time and resources. In general, it is not practically feasible to execute all the possible test cases and so, ordering test case ...

Comments

Information & Contributors

Information

Published In

EuroPLoP '11: Proceedings of the 16th European Conference on Pattern Languages of Programs

July 2011

250 pages

ISBN:9781450313025

DOI:10.1145/2396716

Conference Chair:
Andreas Fiesser,
Program Chair:
Paris Avgeriou

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

software product lines

Qualifiers

Research-article

Conference

EuroPLoP 2011

Sponsor:

Hillside Europe
IT-Szene Munchen

EuroPLoP 2011: 16th European Conference on Pattern Languages of Programs

July 13 - 17, 2011

Irsee, Germany

Acceptance Rates

EuroPLoP '11 Paper Acceptance Rate 15 of 39 submissions, 38%;

Overall Acceptance Rate 216 of 354 submissions, 61%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
157
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Johansen MHaugen ØFleurey F(2011)Properties of realistic feature models make combinatorial testing of product lines feasibleProceedings of the 14th international conference on Model driven engineering languages and systems10.5555/2050655.2050721(638-652)Online publication date: 16-Oct-2011
https://dl.acm.org/doi/10.5555/2050655.2050721
Johansen MHaugen ØFleurey F(2011)Properties of Realistic Feature Models Make Combinatorial Testing of Product Lines FeasibleModel Driven Engineering Languages and Systems10.1007/978-3-642-24485-8_47(638-652)Online publication date: 2011
https://doi.org/10.1007/978-3-642-24485-8_47

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Model-based pairwise testing for feature interaction coverage in software product line engineering

Pairwise testing for software product lines: comparison of two approaches

Multi-objective test prioritization in software product line testing: an industrial case study