research-article

Testing vs. code inspection vs. what else?: male and female end users' debugging strategies

Authors:

Neeraja Subrahmaniyan,

Laura Beckwith,

Valentina Grigoreanu,

Margaret Burnett,

Susan Wiedenbeck,

Vaishnavi Narayanan,

Russell Drummond,

Xiaoli FernAuthors Info & Claims

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Pages 617 - 626

https://doi.org/10.1145/1357054.1357153

Published: 06 April 2008 Publication History

Abstract

Little is known about the strategies end-user programmers use in debugging their programs, and even less is known about gender differences that may exist in these strategies. Without this type of information, designers of end-user programming systems cannot know the "target" at which to aim, if they are to support male and female end-user programmers. We present a study investigating this issue. We asked end-user programmers to debug spreadsheets and to describe their debugging strategies. Using mixed methods, we analyzed their strategies and looked for relationships among participants' strategy choices, gender, and debugging success. Our results indicate that males and females debug in quite different ways, that opportunities for improving support for end-user debugging strategies for both genders are abundant, and that tools currently available to end-user debuggers may be especially deficient in supporting debugging strategies used by females.

References

[1]

Abraham, R., Erwig, M. UCheck: A spreadsheet unit checker for end users, J. Vis. Langs. Comput. 18, 1 (2007), 71--95.

Digital Library

[2]

Anonymous, H. M. Customs and Excise Computer Audit Ser-vice, Methodology for the Audit of Spreadsheet Models, 2001. http://customs.hmrc.gov.uk/channelsPortalWebApp/channelsPortalWebApp.portal?_nfpb=true&_pageLabel= pageVAT_ShowContent&id=HMCE_PROD_009443&propertyType=document (downloaded Aug. 28, 2007).

[3]

Basili, V., Selby, R. Comparing the effectiveness of software testing strategies, IEEE Trans. Soft. Eng. 13, 12 (1987) 1278--1296.

Digital Library

[4]

Beckwith, L. Burnett, M., Wiedenbeck, S., Cook, C., Sorte, S., and Hastings, M. Effectiveness of end-user debugging software features: Are there gender issues? In Proc. CHI 2005, ACM Press (2005), 869--878.

Digital Library

[5]

Beckwith, L. Kissinger, C., Burnett, M., Wiedenbeck, S., Lawrance, J., Blackwell, A., and Cook, C. Tinkering and gender in end-user programmers' debugging, In Proc. CHI 2006, ACM Press (2006), 231--240.

Digital Library

[6]

Beckwith, L., Inman, D., Rector, K., and Burnett, M. On to the real world: Gender and self-efficacy in Excel, In Proc. VLHCC, IEEE (2007).

Digital Library

[7]

Blackwell, A. First steps in programming: a rationale for attention investment models. In Proc. VLHCC, IEEE (2002), 2--10.

Digital Library

[8]

Burnett, M., Cook, C., and Rothermel G. End-user software engineering. Comm. ACM 47, 9 (2004), 53--58.

Digital Library

[9]

Byrnes, J., Miller, C., and Schafer D. Gender differences in risk taking: A meta-analysis. Psych. Bulletin 125 (1999), 367--383.

[10]

Carroll, J. (Ed.), "Minimalism Beyond "The Nurnberg Funnel", MIT Press, Cambridge, MA, 1998.

[11]

Cross, N. Expertise in design: An overview. Design Studies 25, 5 (2004), 427--441.

[12]

Fern, X., Komireddy, C., Burnett, M. Mining interpretable human strategies: A case study, In Proc. ICDM, IEEE (2007).

Digital Library

[13]

Gallagher A., De Lisi R., Holst P., McGillicuddy-De Lisi A., Morely M., Cahalan C. Gender differences in advanced mathematical problem solving, J. Experimental Child Psychology 75, 3 (2000), 165--190.

[14]

Halpern, D. Sex Differences in Cognitive Abilities, 3rd Edition. Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2000.

[15]

Kelleher, C., Pausch, R., and Kiesler, S. Storytelling Alice motivates middle school girls to learn computer programming, In Proc. CHI 2007, ACM Press (2007), 1455--1464.

Digital Library

[16]

Ko, A. and Myers, B. Designing the Whyline: A debugging interface for asking questions about program failures. In Proc. CHI 2004, ACM Press (2004), 151--158.

Digital Library

[17]

Littman, D., Pinto, J., Letovsky, S., and Soloway, E. Mental models and software maintenance. In E. Soloway and S. Iyengar (Eds), In Proc. ESP. Ablex, Norwood, NJ (1986), 80--98.

Digital Library

[18]

Meyers-Levy, J. Gender differences in information processing: A selectivity interpretation. In P. Cafferata & A. Tybout, (Eds) Cognitive and Affective Responses to Advertising. Lexington, Ma, Lexington Books, 1989.

[19]

Nanja, N. and Cook, C. An analysis of the on-line debugging process. In G. M. Olson, S. Sheppard, and E. Soloway (Eds.), In Proc. ESP. Ablex, Norwood, NJ, 1987.

Digital Library

[20]

O'Donnell, E. and Johnson, E. The effects of auditor gender and task complexity on information processing efficiency. Int. J. Auditing 5 (2001), 91--105.

[21]

Powell, S., Baker, K., Lawson, B. An Auditing Protocol for Spreadsheet Models, Jan. 2007. http://mba.tuck.dartmouth.edu/

[22]

spreadsheet/product_pubs.html (downloaded Aug. 28, 2007).

[23]

Rosson, M., Sinha, H., Bhattacharya, M., Zhao, D. Design planning in end-user web development, In Proc. VLHCC, IEEE (2007).

Digital Library

[24]

Subrahmaniyan N., Kissinger, C., Rector, K., Inman, D., Kap-lan, J., Beckwith, L., Burnett, M., Explaining debugging strate-gies to end-user programmers, In Proc. VLHCC, IEEE (2007).

Digital Library

[25]

Wagner, E. and Lieberman, H. Supporting user hypotheses in problem diagnosis on the web and elsewhere. In Proc. IUI, ACM Press (2004), 30--37.

Digital Library

[26]

Weiser, M. Programmers use slices when debugging, Comm. ACM 25, 7 (1982), 446--452.

Digital Library

Cited By

Goldstein HTao JHatfield-Dodds ZPierce BHead A(2024)Tyche: Making Sense of PBT EffectivenessProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676407(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676407
Zhang YPaquette LPinto JLiu QFan A(2022)Combining latent profile analysis and programming traces to understand novices’ differences in debuggingEducation and Information Technologies10.1007/s10639-022-11343-728:4(4673-4701)Online publication date: 22-Oct-2022
https://doi.org/10.1007/s10639-022-11343-7
Scheuerman MJiang ASpiel KBrubaker JKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Revisiting Gendered Web Forms: An Evaluation of Gender Inputs with (Non-)Binary PeopleProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445742(1-18)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445742
Show More Cited By

Index Terms

Testing vs. code inspection vs. what else?: male and female end users' debugging strategies
1. Human-centered computing
  1. Human computer interaction (HCI)
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging

Recommendations

Effectiveness of end-user debugging software features: are there gender issues?
CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Although gender differences in a technological world are receiving significant research attention, much of the research and practice has aimed at how society and education can impact the successes and retention of female computer science professionals-...
Tinkering and gender in end-user programmers' debugging
CHI '06: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Earlier research on gender effects with software features intended to help problem-solvers in end-user debugging environments has shown that females are less likely to use unfamiliar software features. This poses a serious problem because these features ...
End-user debugging strategies: A sensemaking perspective

Despite decades of research into how professional programmers debug, only recently has work emerged about how end-user programmers attempt to debug programs. Without this knowledge, we cannot build tools to adequately support their needs. This article ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '08: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

April 2008

1870 pages

ISBN:9781605580111

DOI:10.1145/1357054

General Chairs:
Mary Czerwinski
Microsoft Research, USA
,
Arnie Lund
Microsoft, USA
,
Program Chair:
Desney Tan
Microsoft Research, USA

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 April 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '08

Sponsor:

CHI '08: CHI Conference on Human Factors in Computing Systems

April 5 - 10, 2008

Florence, Italy

Acceptance Rates

CHI '08 Paper Acceptance Rate 157 of 714 submissions, 22%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI '25

Sponsor:
sigchi

CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
1,042
Total Downloads

Downloads (Last 12 months)23
Downloads (Last 6 weeks)2

Reflects downloads up to 03 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Goldstein HTao JHatfield-Dodds ZPierce BHead A(2024)Tyche: Making Sense of PBT EffectivenessProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676407(1-16)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676407
Zhang YPaquette LPinto JLiu QFan A(2022)Combining latent profile analysis and programming traces to understand novices’ differences in debuggingEducation and Information Technologies10.1007/s10639-022-11343-728:4(4673-4701)Online publication date: 22-Oct-2022
https://doi.org/10.1007/s10639-022-11343-7
Scheuerman MJiang ASpiel KBrubaker JKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Revisiting Gendered Web Forms: An Evaluation of Gender Inputs with (Non-)Binary PeopleProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445742(1-18)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445742
Rodríguez-Pérez GNadri RNagappan M(2021)Perceived diversity in software engineering: a systematic literature reviewEmpirical Software Engineering10.1007/s10664-021-09992-226:5Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1007/s10664-021-09992-2
Bai GKayani JStolee K(2020)How Graduate Computing Students Search When Using an Unfamiliar Programming LanguageProceedings of the 28th International Conference on Program Comprehension10.1145/3387904.3389274(160-171)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.1145/3387904.3389274
Huang YLeach KSharafi ZMcKay NSantander TWeimer WDevanbu PCohen MZimmermann T(2020)Biases and differences in code review using medical imaging and eye-tracking: genders, humans, and machinesProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3368089.3409681(456-468)Online publication date: 8-Nov-2020
https://dl.acm.org/doi/10.1145/3368089.3409681
Vorvoreanu MZhang LHuang YHilderbrand CSteine-Hanson ZBurnett MBrewster SFitzpatrick GCox AKostakos V(2019)From Gender Biases to Gender-Inclusive DesignProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300283(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300283
Tzafilkou KProtogeros NKaragiannidis CKoumpis A(2017)Gender-based behavioral analysis for end-user development and the `RULES' attributesEducation and Information Technologies10.5555/3123587.312359522:4(1853-1894)Online publication date: 1-Jul-2017
https://dl.acm.org/doi/10.5555/3123587.3123595
Spichkova MSchmidt HTrubiani Cde Lemos R(2017)Role of women in software architectureProceedings of the 11th European Conference on Software Architecture: Companion Proceedings10.1145/3129790.3129826(31-34)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3129790.3129826
Tzafilkou KProtogeros NKoumpis A(2017)User-centred cloud service adaptationInternational Journal of Computer Integrated Manufacturing10.1080/0951192X.2015.103069730:4-5(472-482)Online publication date: 1-May-2017
https://dl.acm.org/doi/10.1080/0951192X.2015.1030697
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents