research-article

Aggregate Compilation Behavior: Findings and Implications from 27,698 Users

Authors:

Matthew C. Jadud,

Brian DornAuthors Info & Claims

ICER '15: Proceedings of the eleventh annual International Conference on International Computing Education Research

Pages 131 - 139

https://doi.org/10.1145/2787622.2787718

Published: 09 August 2015 Publication History

Abstract

The error quotient (EQ) was first reported in 2006 as a behavioral measure of novice programmers. The EQ scores how well students deal with correcting syntax errors (or not) in their programs. The original studies were carried out on data collected using BlueJ, a pedagogic Java programming environment; today, newly installed instances of BlueJ capture data similar to these early studies automatically, meaning data regarding nearly 2 million programmers is captured every year by the Blackbox project. In this paper, we apply Jadud's original error quotient algorithm to this new, massive data set, and discuss our results and analysis in light of related work. Further, we consider the implications of our findings for researchers and educators in applying the EQ to 27,698 users in 10 different countries during the fall term of 2013.

References

[1]

J. Albert and R. H. Koning. Statistical thinking in sports. CRC Press, 2007.

[2]

A. Altadmri and N. C. Brown. 37 million compilations: Investigating novice programming mistakes in large-scale student data. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education, SIGCSE '15, pages 522--527, New York, NY, USA, 2015. ACM.

Digital Library

[3]

D. J. Barnes and M. Kölling. Objects First with Java: A Practical Introduction Using BlueJ. Prentice Hall Press, Upper Saddle River, NJ, USA, 5th edition, 2011.

Digital Library

[4]

N. C. Brown and A. Altadmri. Investigating novice programming mistakes: Educator beliefs vs. student data. In Proceedings of the Tenth Annual Conference on International Computing Education Research, ICER '14, pages 43--50, 2014.

Digital Library

[5]

N. C. C. Brown, M. Kölling, D. McCall, and I. Utting. Blackbox: A large scale repository of novice programmers' activity. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education, SIGCSE '14, pages 223--228, 2014.

Digital Library

[6]

S. H. Edwards and M. A. Perez-Quinones. Web-cat: automatically grading programming assignments. In ACM SIGCSE Bulletin, volume 40, pages 328--328. ACM, 2008.

Digital Library

[7]

T. R. Green. Cognitive dimensions of notations. A. Sutcliffe and Macaulay, editors, People and Computers V, pages 443--460, 1989.

Digital Library

[8]

T. R. G. Green and M. Petre. Usability analysis of visual programming environments: a 'cognitive dimensions' framework. Journal of Visual Languages & Computing, 7(2):131--174, 1996.

[9]

M. C. Jadud. Methods and tools for exploring novice compilation behaviour. In Proceedings of the Second International Workshop on Computing Education Research, ICER '06, pages 73--84, 2006.

Digital Library

[10]

M. Kölling, B. Quig, A. Patterson, and J. Rosenberg. The bluej system and its pedagogy. Computer Science Education, 13(4):249--268, 2003.

[11]

K. Pearson. Contributions to the mathematical theory of evolution. ii. skew variation in homogeneous material. Philosophical Transactions of the Royal Society of London. A, pages 343--414, 1895.

[12]

R Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2013.

[13]

M. M. T. Rodrigo, T. C. S. Andallaza, F. E. V. G. Castro, M. L. V. Armenta, T. T. Dy, and M. C. Jadud. An analysis of java programming behaviors, affect, perceptions, and syntax errors among low-achieving, average, and high-achieving novice programmers. Journal of Educational Computing Research, 49(3):293--325, 2013.

[14]

M. M. T. Rodrigo, R. S. Baker, M. C. Jadud, A. C. M. Amarra, T. Dy, M. B. V. Espejo-Lahoz, S. A. L. Lim, S. A. Pascua, J. O. Sugay, and E. S. Tabanao. Affective and behavioral predictors of novice programmer achievement. In Proceedings of the 14th Annual ACM SIGCSE Conference on Innovation and Technology in Computer Science Education, ITiCSE '09, pages 156--160, New York, NY, USA, 2009. ACM.

Digital Library

[15]

A. Stefik and S. Siebert. An empirical investigation into programming language syntax. ACM Transactions on Computing Education (TOCE), 13(4):19, 2013.

Digital Library

[16]

E. S. Tabanao, M. M. T. Rodrigo, and M. C. Jadud. Predicting at-risk novice java programmers through the analysis of online protocols. In Proceedings of the Seventh International Workshop on Computing Education Research, ICER '11, pages 85--92, 2011.

Digital Library

[17]

C. Watson, F. W. Li, and J. L. Godwin. Predicting performance in an introductory programming course by logging and analyzing student programming behavior. In Advanced Learning Technologies (ICALT), 2013 IEEE 13th International Conference on, pages 319--323. IEEE, 2013.

Digital Library

[18]

C. Watson, F. W. Li, and J. L. Godwin. No tests required: Comparing traditional and dynamic predictors of programming success. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education, SIGCSE '14, pages 469--474, 2014.

Digital Library

Cited By

Kaur HKaur KKaur K(2024)Enhancing student experience in remote computer programming course practice: A case of the Java languageE-Learning and Digital Media10.1177/20427530241262485Online publication date: 14-Jun-2024
https://doi.org/10.1177/20427530241262485
Švábenský VPankiewicz MZhang JCloude EBaker RFouh EMonga MLonati VBarendsen ESheard JPaterson J(2024)Comparison of Three Programming Error Measures for Explaining Variability in CS1 GradesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653563(87-93)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653563
Qian YWu Y(2024)Exploring the Effectiveness of Data-Driven Learning Materials for Promoting Student Performance in Introductory ProgrammingIEEE Access10.1109/ACCESS.2024.345464312(125170-125178)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3454643
Show More Cited By

Index Terms

Aggregate Compilation Behavior: Findings and Implications from 27,698 Users
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education

Recommendations

Blackbox: a large scale repository of novice programmers' activity
SIGCSE '14: Proceedings of the 45th ACM technical symposium on Computer science education

Automatically observing and recording the programming behaviour of novices is an established computing education research technique. However, prior studies have been conducted at a single institution on a small or medium scale, without the possibility ...
Methods and tools for exploring novice compilation behaviour
ICER '06: Proceedings of the second international workshop on Computing education research

Our research explores what we call compilation behaviour: the programming behaviour a student engages in while repeatedly editing and compiling their programs. This edit-compile cycle often represents students' attempts to make their programs ...
The Effects of Compilation Mechanisms and Error Message Presentation on Novice Programmer Behavior
SIGCSE '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education

It is generally accepted that learning to program could be easier for many students. One of the most important components of this experience is the programming environment. Novices learn in a variety of environments, from basic command-line interfaces ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICER '15: Proceedings of the eleventh annual International Conference on International Computing Education Research

July 2015

300 pages

ISBN:9781450336307

DOI:10.1145/2787622

General Chair:
Brian Dorn
University of Nebraska at Omaha, USA
,
Program Chairs:
Judy Sheard
Monash University, Australia
,
Quintin Cutts
University of Glasgow, UK

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 August 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

ICER '15

Sponsor:

SIGCSE

ICER '15: International Computing Education Research Conference

July 9 - 13, 2015

Nebraska, Omaha, USA

Acceptance Rates

ICER '15 Paper Acceptance Rate 25 of 96 submissions, 26%;

Overall Acceptance Rate 189 of 803 submissions, 24%

Upcoming Conference

ICER 2025

Sponsor:
sigcse

ACM Conference on International Computing Education Research

August 3 - 6, 2025

Charlottesville , VA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

33
Total Citations
View Citations
522
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Kaur HKaur KKaur K(2024)Enhancing student experience in remote computer programming course practice: A case of the Java languageE-Learning and Digital Media10.1177/20427530241262485Online publication date: 14-Jun-2024
https://doi.org/10.1177/20427530241262485
Švábenský VPankiewicz MZhang JCloude EBaker RFouh EMonga MLonati VBarendsen ESheard JPaterson J(2024)Comparison of Three Programming Error Measures for Explaining Variability in CS1 GradesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653563(87-93)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653563
Qian YWu Y(2024)Exploring the Effectiveness of Data-Driven Learning Materials for Promoting Student Performance in Introductory ProgrammingIEEE Access10.1109/ACCESS.2024.345464312(125170-125178)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3454643
Pankiewicz MZambrano ABarany ABaker R(2024)How We Code Code: Leveraging GPT and Ordered Networks for Understanding Introductory Programming EducationAdvances in Quantitative Ethnography10.1007/978-3-031-76332-8_19(225-240)Online publication date: 2-Nov-2024
https://doi.org/10.1007/978-3-031-76332-8_19
Cloude EBaker RFouh EHilliger IKhosravi HRienties BDawson S(2023)Online help-seeking occurring in multiple computer-mediated conversations affects grades in an introductory programming courseLAK23: 13th International Learning Analytics and Knowledge Conference10.1145/3576050.3576106(378-387)Online publication date: 13-Mar-2023
https://dl.acm.org/doi/10.1145/3576050.3576106
Zhang YZhu LJiang GKwong SKuo C(2023)A Survey on Perceptually Optimized Video CodingACM Computing Surveys10.1145/357172755:12(1-37)Online publication date: 2-Mar-2023
https://dl.acm.org/doi/10.1145/3571727
Silva LGomes AMendes A(2023)Exploring the Impact of Self-Regulation of Learning Support on Programming Performance and Code Development2023 IEEE Frontiers in Education Conference (FIE)10.1109/FIE58773.2023.10343321(1-9)Online publication date: 18-Oct-2023
https://doi.org/10.1109/FIE58773.2023.10343321
Chandarana MDeitrick EKizilcec RDavis KOchoa X(2022)Challenges of Scaling Programming-based Behavioral MetricsProceedings of the Ninth ACM Conference on Learning @ Scale10.1145/3491140.3528290(304-308)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1145/3491140.3528290
Karvelas IBecker BMerkle LDoyle MSheard JSoh LDorn B(2022)Sympathy for the (Novice) DeveloperProceedings of the 53rd ACM Technical Symposium on Computer Science Education - Volume 110.1145/3478431.3499347(962-968)Online publication date: 22-Feb-2022
https://dl.acm.org/doi/10.1145/3478431.3499347
Poulsen SButler LAlawini AHerman GGiannakos MSindre GLuxton-Reilly ADivitini M(2020)Insights from Student Solutions to SQL Homework ProblemsProceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education10.1145/3341525.3387391(404-410)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3341525.3387391
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents