research-article

How to compare program comprehension in FOSD empirically: an experience report

Authors:

Janet Feigenspan,

Christian Kästner,

Thomas LeichAuthors Info & Claims

FOSD '09: Proceedings of the First International Workshop on Feature-Oriented Software Development

Pages 55 - 62

https://doi.org/10.1145/1629716.1629728

Published: 06 October 2009 Publication History

Abstract

There are many different implementation approaches to realize the vision of feature-oriented software development, ranging from simple preprocessors, over feature-oriented programming, to sophisticated aspect-oriented mechanisms. Their impact on readability and maintainability (or program comprehension in general) has caused a debate among researchers, but sound empirical results are missing. We report experience from our endeavor to conduct experiments to measure the influence of different implementation mechanisms on program comprehension. We describe how to design such experiments and report from possibilities and pitfalls we encountered. Finally, we present some early results of our first experiment on comparing the CPP tool with the CIDE tool.

References

[1]

J. R. Anderson. Cognitive Psychology and its Implications. Worth Publishers, 2000.

[2]

T. W. Anderson and J. D. Finn. The New Statistical Analysis of Data. Springer, 1996.

[3]

S. Apel et al. FeatureC++: On the Symbiosis of Feature-Oriented and Aspect-Oriented Programming. In Proc. Int'l Conf. Generative Programming and Component Engineering, pages 125--140, 2005.

Digital Library

[4]

S. Apel and C. Kästner. An Overview of Feature-Oriented Software Development. Journal of Object Technology (JOT), 8(5):49--84, 2009.

[5]

S. Apel, C. Kästner, and S. Trujillo. On the Necessity of Empirical Studies in the Assessment of Modularization Mechanisms for Crosscutting Concerns. In Proc. Int'l Workshop on Assessment of Contemporary Modularization Techniques, pages 1--7. 2007.

Digital Library

[6]

I. Aracic, V. Gasiunas, M. Mezini, and K. Ostermann. An Overview of CaesarJ. Transactions on Aspect-Oriented Software Development 1, pages 135--173, 2006.

Digital Library

[7]

A. P. Association. Publication Manual of the American Psychological Association. American Psychological Association, 2001.

[8]

D. Batory, J. N. Sarvela, and A. Rauschmayer. Scaling Step-Wise Refinement. IEEE Trans. Softw. Eng., 30(6):355--371, 2004.

Digital Library

[9]

R. E. Brooks. Using a Behavioral Theory of Program Comprehension in Software Engineering. In Proc. Int'l Conf. Software Engineering, pages 196--201. 1978.

Digital Library

[10]

J. Daly et al. The Effect of Inheritance on the Maintainability of Object-Oriented Software: An Empirical Study. In Proc. Int'l Conf. Software Maintenance, pages 20--29. 1995.

Digital Library

[11]

A. Dunsmore and M. Roper. A Comparative Evaluation of Program Comprehension Measures. Technical Report EFoCS-35-2000, University of Strathclyde, 2000.

[12]

S. Easterbrook, J. Singer, M.-A. Storey, and D. Damian. Guide to Advanced Empirical Software Engineering, pages 285--311. Springer, 2008.

[13]

J. Favre. Understanding-In-The-Large. In Proc. Int'l Workshop on Program Comprehension, page 29. 1997.

Digital Library

[14]

J. Feigenspan. Empirical Comparison of FOSD Approaches Regarding Program Comprehension -- A Feasibility Study. Master's thesis, University of Magdeburg, 2009.

[15]

E. Figueiredo et al. Evolving Software Product Lines with Aspects: An Empirical Study on Design Stability. In Proc. Int'l Conf. Software Engineering, pages 261--270. 2008.

Digital Library

[16]

C. J. Goodwin. Research In Psychology: Methods and Design. Wiley Publishing, Inc., 1998.

[17]

S. Hanenberg, S. Kleinschmager, and M. Josupeit-Walter. Does Aspect-Oriented Programming Increase The Development Speed for Crosscutting Code? An Empirical Study. In Proc. Int'l Symp. Empirical Software Engineering and Measurement, 2009.

Digital Library

[18]

S. Henry, M. Humphrey, and J. Lewis. Evaluation of the Maintainability of Object-Oriented Software. In Proc. TENCON, pages 404--409. 1990.

[19]

A. Höfer and W. Tichy. Empirical Software Engineering Issues. Critical Assessment and Future Directions, chapter Status of Empirical Research in Software Engineering, pages 10--19. Springer, 2007.

Digital Library

[20]

International Organization for Standardization. ISO/IEC 9899-1999: Programming Languages---C, 1999.

[21]

S. Jarzabek, P. Bassett, H. Zhang, and W. Zhang. XVCL: XML-based Variant Configuration Language. In Proc. Int'l Conf. Software Engineering, pages 810--811. 2003.

Digital Library

[22]

N. Juristo and A. M. Moreno. Basics of Software Engineering Experimentation. Kluwer, 2001.

Digital Library

[23]

V. B. Kampenes et al. A Systematic Review of Quasi-Experiments in Software Engineering. Information and Software Technology, 51(1):71--82, 2009.

Digital Library

[24]

C. Kästner, S. Apel, and M. Kuhlemann. Granularity in Software Product Lines. In Proc. Int'l Conf. Software Engineering, pages 311--320. 2008.

Digital Library

[25]

C. Kästner, S. Trujillo, and S. Apel. Visualizing Software Product Line Variabilities in Source Code. In Proc. Workshop Visualization in Software Product Line Engineering, 2008.

[26]

G. Kiczales et al. Aspect-Oriented Programming. In Proc. Europ. Conf. Object-Oriented Programming, pages 220--242. 1997.

[27]

G. Kiczales et al. An Overview of AspectJ. In Proc. Europ. Conf. Object-Oriented Programming, pages 327--353. 2001.

Digital Library

[28]

J. Koenemann and S. P. Robertson. Expert Problem Solving Strategies for Program Comprehension. In Proc. Conf. Human Factors in Computing Systems, pages 125--130. 1991.

Digital Library

[29]

B. P. Lientz and E. B. Swanson. Software Maintenance Management. Addison-Wesley, 1980.

Digital Library

[30]

R. Lopez-Herrejon, D. Batory, and W. Cook. Evaluating Support for Features in Advanced Modularization Technologies. In Proc. Europ. Conf. Object-Oriented Programming, pages 169--194. 2005.

Digital Library

[31]

M. Mezini and K. Ostermann. Variability Management with Feature-Oriented Programming and Aspects. In Proc. Int'l Symp. Foundations of Software Engineering, pages 127--136. 2004.

Digital Library

[32]

S. Patig. A Practical Guide to Testing the Understandability of Notations. In Proc. Asia-Pacific Conf. Conceptual Modelling, pages 49--58, 2008.

Digital Library

[33]

N. Pennington. Stimulus Structures and Mental Representations in Expert Comprehension of Computer Programs. Cognitive Psychologys, 19(3):295--341, 1987.

[34]

K. Popper. The Logic of Scientic Discovery. Routledge, 1959.

[35]

L. Prechelt et al. Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance. IEEE Trans. Softw. Eng., 28(6):595--606, 2002.

Digital Library

[36]

C. Prehofer. Feature-Oriented Programming: A Fresh Look at Objects. In Proc. Europ. Conf. Object-Oriented Programming, pages 419--443. 1997.

[37]

F. J. Roethlisberger. Management and the Worker. Harvard University Press, 1939.

[38]

R. Rosenthal and L. Jacobson. Teachers' Expectancies: Determinants of Pupils' IQ Gains. Psychological Reports, 19(1):115--118, 1966.

[39]

W. R. Shadish, T. D. Cook, and D. T. Campbell. Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, 2002.

[40]

D. I. K. Sjoberg et al. A Survey of Controlled Experiments in Software Engineering. IEEE Trans. Softw. Eng., 31(9):733--753, 2005.

Digital Library

[41]

H. Spencer and G. Collyer. #ifdef Considered Harmful or Portability Experience With C News. In Proc. USENIX Conf., pages 185--198, 1992.

[42]

A. von Mayrhauser and A. M. Vans. Program Comprehension During Software Maintenance and Evolution. Computer, 28(8):44--55, 1995.

Digital Library

Cited By

Krüger JÇalıklı GBerger TLeich TSaake GDumas MPfahl DApel SRusso A(2019)Effects of explicit feature traceability on program comprehensionProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3338906.3338968(338-349)Online publication date: 12-Aug-2019
https://dl.acm.org/doi/10.1145/3338906.3338968
Rodrigues Santos ACarmo Machado ISantana De Almeida ESiegmund JApel S(2019)Comparing the influence of using feature-oriented programming and conditional compilation on comprehending feature-oriented softwareEmpirical Software Engineering10.1007/s10664-018-9658-x24:3(1226-1258)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s10664-018-9658-x
Krüger JHaddad HWainwright RChbeir R(2018)Separation of concernsProceedings of the 33rd Annual ACM Symposium on Applied Computing10.1145/3167132.3167458(2076-2077)Online publication date: 9-Apr-2018
https://dl.acm.org/doi/10.1145/3167132.3167458
Show More Cited By

Index Terms

How to compare program comprehension in FOSD empirically: an experience report
1. Software and its engineering
  1. Software creation and management
    1. Designing software
      1. Software implementation planning
        Software design techniques
    2. Software development process management
  2. Software notations and tools
    1. General programming languages
      1. Language features

Recommendations

Do background colors improve program comprehension in the #ifdef hell?

Software-product-line engineering aims at the development of variable and reusable software systems. In practice, software product lines are often implemented with preprocessors. Preprocessor directives are easy to use, and many mature tools are ...
FeatureIDE: An extensible framework for feature-oriented software development

FeatureIDE is an open-source framework for feature-oriented software development (FOSD) based on Eclipse. FOSD is a paradigm for the construction, customization, and synthesis of software systems. Code artifacts are mapped to features, and a customized ...
Feature interactions, products, and composition
GCPE '11

The relationship between feature modules and feature interactions is not well-understood. To explain classic examples of feature interaction, we show that features are not only composed sequentially, but also by cross-product and interaction operations ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

FOSD '09: Proceedings of the First International Workshop on Feature-Oriented Software Development

October 2009

127 pages

ISBN:9781605585673

DOI:10.1145/1629716

Editors:
Sven Apel
University of Passau, DE
,
William R. Cook
University of Texas at Austin
,
Krzysztof Czarnecki
University of Waterloo, CA
,
Christian Kastner
University of Magdeburg, DE
,
Neil Loughran
SINTEF, NO
,
Oscar Nierstrasz
University of Berne, CH

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Metop GmbH

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 October 2009

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

FOSD '09

Sponsor:

FOSD '09: 1st International Workshop on Feature-Oriented Software Development

October 6, 2009

Colorado, Denver, USA

Acceptance Rates

Overall Acceptance Rate 17 of 28 submissions, 61%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
247
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Krüger JÇalıklı GBerger TLeich TSaake GDumas MPfahl DApel SRusso A(2019)Effects of explicit feature traceability on program comprehensionProceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3338906.3338968(338-349)Online publication date: 12-Aug-2019
https://dl.acm.org/doi/10.1145/3338906.3338968
Rodrigues Santos ACarmo Machado ISantana De Almeida ESiegmund JApel S(2019)Comparing the influence of using feature-oriented programming and conditional compilation on comprehending feature-oriented softwareEmpirical Software Engineering10.1007/s10664-018-9658-x24:3(1226-1258)Online publication date: 1-Jun-2019
https://dl.acm.org/doi/10.1007/s10664-018-9658-x
Krüger JHaddad HWainwright RChbeir R(2018)Separation of concernsProceedings of the 33rd Annual ACM Symposium on Applied Computing10.1145/3167132.3167458(2076-2077)Online publication date: 9-Apr-2018
https://dl.acm.org/doi/10.1145/3167132.3167458
Krüger JLudwig KZimmermann BLeich THaddad HWainwright RChbeir R(2018)Physical separation of featuresProceedings of the 33rd Annual ACM Symposium on Applied Computing10.1145/3167132.3167351(2042-2049)Online publication date: 9-Apr-2018
https://dl.acm.org/doi/10.1145/3167132.3167351
Krüger JSchröter IKenner AKruczek CLeich TSeidl CTeixeira L(2016)FeatureCoPP: compositional annotationsProceedings of the 7th International Workshop on Feature-Oriented Software Development10.1145/3001867.3001876(74-84)Online publication date: 30-Oct-2016
https://dl.acm.org/doi/10.1145/3001867.3001876
Lee HKang KGnesi SCollet PSchmid K(2013)A design feature-based approach to deriving program code from featuresProceedings of the 7th International Workshop on Variability Modelling of Software-Intensive Systems10.1145/2430502.2430510(1-6)Online publication date: 23-Jan-2013
https://dl.acm.org/doi/10.1145/2430502.2430510
Feigenspan JSiegmund N(2012)Supporting comprehension experiments with human subjects2012 20th IEEE International Conference on Program Comprehension (ICPC)10.1109/ICPC.2012.6240494(244-246)Online publication date: Jun-2012
https://doi.org/10.1109/ICPC.2012.6240494
Feigenspan JSchulze MPapendieck MKästner CDachselt RKöppen VFrisch MSaake G(2012)Supporting program comprehension in large preprocessor-based software product linesIET Software10.1049/iet-sen.2011.01726:6(488-501)Online publication date: 14-Dec-2012
https://doi.org/10.1049/iet-sen.2011.0172
Siegmund JSiegmund NFruth JKuhlmann SDittmann JSaake G(2012)Program comprehension in preprocessor-based softwareProceedings of the 2012 international conference on Computer Safety, Reliability, and Security10.1007/978-3-642-33675-1_49(517-528)Online publication date: 25-Sep-2012
https://dl.acm.org/doi/10.1007/978-3-642-33675-1_49
Stengel MFrisch MApel SFeigenspan JKästner CDachselt RTaylor RGall HMedvidović N(2011)View infinityProceedings of the 33rd International Conference on Software Engineering10.1145/1985793.1985987(1031-1033)Online publication date: 21-May-2011
https://dl.acm.org/doi/10.1145/1985793.1985987
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents