Article

How Well Do Professional Developers Test with Code Coverage Visualizations? An Empirical Study

Authors:

Joseph Lawrance,

Steven Clarke,

Margaret Burnett,

Gregg RothermelAuthors Info & Claims

VLHCC '05: Proceedings of the 2005 IEEE Symposium on Visual Languages and Human-Centric Computing

Pages 53 - 60

https://doi.org/10.1109/VLHCC.2005.44

Published: 20 September 2005 Publication History

Publisher Site

Abstract

Despite years of availability of testing tools, professional software developers still seem to need better support to determine the effectiveness of their tests. Without improvements in this area, inadequate testing of software seems likely to remain a major problem. To address this problem, industry and researchers have proposed systems that visualize "testedness" for end-user and professional developers. Empirical studies of such systems for end-user programmers have begun to show success at helping end users write more effective tests. Encouraged by this research, we examined the effect that code coverage visualizations have on the effectiveness of test cases that professional software developers write. This paper presents the results of an empirical study conducted using code coverage visualizations found in a commercially available programming environment. Our results reveal how this kind of code coverage visualization impacts test effectiveness, and provide insights into the strategies developers use to test code.

Cited By

View all

Openja MKhomh FFoundjem AJiang ZAbidi MHassan A(2024)An Empirical Study of Testing Machine Learning in the WildACM Transactions on Software Engineering and Methodology10.1145/368046334:1(1-63)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1145/3680463
Goldstein HTao JHatfield-Dodds ZPierce BHead A(2024)Tyche: Making Sense of PBT EffectivenessProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676407(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676407
Vidoni M(2021)Evaluating Unit Testing Practices in R PackagesProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00136(1523-1534)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00136
Show More Cited By

Index Terms

How Well Do Professional Developers Test with Code Coverage Visualizations? An Empirical Study
1. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software notations and tools
    1. Context specific languages
      1. Visual languages

Recommendations

An empirical study of bugs in test code
ICSME '15: Proceedings of the 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME)

Testing aims at detecting (regression) bugs in production code. However, testing code is just as likely to contain bugs as the code it tests. Buggy test cases can silently miss bugs in the production code or loudly ring false alarms when the production ...
An empirical study on the influence of developers’ experience on software test code quality
SBQS '22: Proceedings of the XXI Brazilian Symposium on Software Quality

Software developers’ engagement in open-source software projects lies in different levels of participation, e.g., core or peripheral developers. Recent studies have investigated the role of developers’ contributions and their influence on software ...
An empirical study on effects of code visibility on code coverage of software testing
AST '15: Proceedings of the 10th International Workshop on Automation of Software Test

Software testability is the degree of difficulty to test a program. Code visibility is important to support design principles, such as information hiding. It is widely believed that code visibility has effects on testability. However, little empirical ...

Comments

Information & Contributors

Information

Published In

VLHCC '05: Proceedings of the 2005 IEEE Symposium on Visual Languages and Human-Centric Computing

September 2005

304 pages

ISBN:0769524435

Publisher

IEEE Computer Society

United States

Publication History

Published: 20 September 2005

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Openja MKhomh FFoundjem AJiang ZAbidi MHassan A(2024)An Empirical Study of Testing Machine Learning in the WildACM Transactions on Software Engineering and Methodology10.1145/368046334:1(1-63)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1145/3680463
Goldstein HTao JHatfield-Dodds ZPierce BHead A(2024)Tyche: Making Sense of PBT EffectivenessProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676407(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676407
Vidoni M(2021)Evaluating Unit Testing Practices in R PackagesProceedings of the 43rd International Conference on Software Engineering10.1109/ICSE43902.2021.00136(1523-1534)Online publication date: 22-May-2021
https://dl.acm.org/doi/10.1109/ICSE43902.2021.00136
Motwani MJuristo N(2021)High-quality automated program repairProceedings of the 43rd International Conference on Software Engineering: Companion Proceedings10.1109/ICSE-Companion52605.2021.00134(309-314)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1109/ICSE-Companion52605.2021.00134
Ko AAbraham RBeckwith LBlackwell ABurnett MErwig MScaffidi CLawrance JLieberman HMyers BRosson MRothermel GShaw MWiedenbeck S(2011)The state of the art in end-user software engineeringACM Computing Surveys10.1145/1922649.192265843:3(1-44)Online publication date: 29-Apr-2011
https://dl.acm.org/doi/10.1145/1922649.1922658
Xie TTillmann Nde Halleux JSchulte WRoman GSullivan K(2010)Future of developer testingProceedings of the FSE/SDP workshop on Future of software engineering research10.1145/1882362.1882447(415-420)Online publication date: 7-Nov-2010
https://dl.acm.org/doi/10.1145/1882362.1882447
Subrahmaniyan NBurnett MBogart CKoschke RHundhausen CTelea A(2008)Software visualization for end-user programmersProceedings of the 4th ACM symposium on Software visualization10.1145/1409720.1409742(135-144)Online publication date: 16-Sep-2008
https://dl.acm.org/doi/10.1145/1409720.1409742
Ricca FDi Penta MTorchiano MTonella PCeccato M(2007)The Role of Experience and Ability in Comprehension Tasks Supported by UML StereotypesProceedings of the 29th international conference on Software Engineering10.1109/ICSE.2007.86(375-384)Online publication date: 24-May-2007
https://dl.acm.org/doi/10.1109/ICSE.2007.86
Berner SWeber RKeller R(2007)Enhancing Software Testing by Judicious Use of Code Coverage InformationProceedings of the 29th international conference on Software Engineering10.1109/ICSE.2007.34(612-620)Online publication date: 24-May-2007
https://dl.acm.org/doi/10.1109/ICSE.2007.34

Abstract

Cited By

Index Terms

Recommendations

An empirical study of bugs in test code

An empirical study on the influence of developers’ experience on software test code quality

An empirical study on effects of code visibility on code coverage of software testing

Comments

Information

Published In

Publisher

Publication History

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations