Article

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students

Authors:

Yifat Ben-David Kolikant,

Cary Laxer,

Lynda Thomas,

Ian Utting,

Tadeusz WiluszAuthors Info & Claims

ITiCSE-WGR '01: Working group reports from ITiCSE on Innovation and technology in computer science education

Pages 125 - 180

https://doi.org/10.1145/572133.572137

Published: 01 December 2001 Publication History

Get Access

Abstract

In computer science, an expected outcome of a student's education is programming skill. This working group investigated the programming competency students have as they complete their first one or two courses in computer science. In order to explore options for assessing students, the working group developed a trial assessment of whether students can program. The underlying goal of this work was to initiate dialog in the Computer Science community on how to develop these types of assessments. Several universities participated in our trial assessment and the disappointing results suggest that many students do not know how to program at the conclusion of their introductory courses. For a combined sample of 216 students from four universities, the average score was 22.89 out of 110 points on the general evaluation criteria developed for this study. From this trial assessment we developed a framework of expectations for first-year courses and suggestions for further work to develop more comprehensive assessments.

References

[1]

ACM & IEEE-CS Joint Task Force on Computing Curricula 2001 (2001). Computing Curricula 2001, Ironman Draft. Association for Computing Machinery and the Computer Society of the Institute of Electrical and Electronics Engineers. Available: http://www.acm.org/sigcse/cc2001 {2001, 5/16/01}.

Google Scholar

[2]

Beck, K. (2000). Xtreme Programming Explained: Embrace the Change. The XP Series, Addison-Wesley, 2000, Boston.

Digital Library

Google Scholar

[3]

BlueJ (2001). BlueJ, the Interactive Java Environment. Available: http://www.bluej.org. {24 July 2001}.

Google Scholar

[4]

Hambleton, R. K. (1996). Advances in Assessment Models, Methods, and Practices. In D. C. Berliner and R. C. Calfee (Eds.) Handbook of Educational Psychology. New York: Simon & Schuster Macmillan.

Google Scholar

[5]

Linn, R. L., Baker. E. L., and Dunbar, S. B. (1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20(8), pp. 15-21.

Crossref

Google Scholar

[6]

Mayer, R. E. (1981). A psychology of how novices learn computer programming. Computing Surveys, 1, pp. 121-141.

Digital Library

Google Scholar

[7]

Pea, R. (1986). Language independent conceptual bugs in novice programming. Educational Computing Research, 2(1), pp. 25-36.

Crossref

Google Scholar

[8]

Soloway, E., Ehrlich, K., Bonar, J., & Greenspan, J. (1982). What do novices know about programming? In A. Badre and B. Shneiderman (Eds) Directions in Human-Computer Interactions, Norwood, NJ: Ablex, pp. 27-54.

Google Scholar

[9]

Spohrer, J., & Soloway, E. (1986). Novice mistakes: Are the folk wisdoms correct? Communications of the ACM, 29(7), pp. 624-632.

Digital Library

Google Scholar

[10]

Wiedenbeck, S., Ramalingam, V., Sarasamma, S. and Corritore, C. L. (1999). A comparison of the comprehension of object-oriented and procedural programs by novice programmers. Interacting With Computers. 11(3), March, pp. 255-282.

Crossref

Google Scholar

[11]

Wilson, B. C., & Shrock, S. (2001). Contributing to success in an introductory computer science course: A study of twelve factors. In I. Russell (Ed.), The Proceedings of the Thirty-second SIGCSE Technical Symposium on Computer Science Education. In SIGCSE Bulletin inroads. 33(1). pp. 184-188

Digital Library

Google Scholar

Cited By

View all

Gardella NPettit RRiggs SMonga MLonati VBarendsen ESheard JPaterson J(2024)Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code GenerationProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653615(290-296)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653615
Haglund PMannila LStrömbäck FBerglund AMonga MLonati VBarendsen ESheard JPaterson J(2024)Grasping the Unseen: TA Insights into Teaching Subtle Concepts in Computer ScienceProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653601(157-163)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653601
Lehtinen TKoutcheme CHellas ARoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension QuestionsProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640058(221-232)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640058
Show More Cited By

Index Terms

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Computer science education
      2. Model curricula

Recommendations

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students

In computer science, an expected outcome of a student's education is programming skill. This working group investigated the programming competency students have as they complete their first one or two courses in computer science. In order to explore ...
A first-year experience report on a model for statewide deployment of CS principles courses (abstract only)
SIGCSE '14: Proceedings of the 45th ACM technical symposium on Computer science education

This poster describes our first-year experience in developing and evaluating a CS Principles professional development model for training a cohort of teachers across an entire state geography. The scalable deployment and sustainable persistence of the ...
Insights into Finnish first-year pre-service teachers' twenty-first century skills

This study focuses on Finnish pre-service teachers' perceptions of their twenty-first century skills, especially their learning strategies, collaboration and teamwork, as well as knowledge and attitudes related to ICT in education. The target group ...

Comments

Information & Contributors

Information

Published In

ITiCSE-WGR '01: Working group reports from ITiCSE on Innovation and technology in computer science education

December 2001

140 pages

ISBN:9781450373593

DOI:10.1145/572133

Conference Chair:
Henry M. Walker
Grinnell College, Grinnell, Iowa

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2001

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ITICSE01

Sponsor:

SIGCSE

ITICSE01: The 6th Annual Conference on Innovation and Technology in Computer Science Education

01 12 2001

Canterbury, UK

Acceptance Rates

Overall Acceptance Rate 552 of 1,613 submissions, 34%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

156
Total Citations
View Citations
3,439
Total Downloads

Downloads (Last 12 months)202
Downloads (Last 6 weeks)22

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Gardella NPettit RRiggs SMonga MLonati VBarendsen ESheard JPaterson J(2024)Performance, Workload, Emotion, and Self-Efficacy of Novice Programmers Using AI Code GenerationProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653615(290-296)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653615
Haglund PMannila LStrömbäck FBerglund AMonga MLonati VBarendsen ESheard JPaterson J(2024)Grasping the Unseen: TA Insights into Teaching Subtle Concepts in Computer ScienceProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653601(157-163)Online publication date: 3-Jul-2024
https://dl.acm.org/doi/10.1145/3649217.3653601
Lehtinen TKoutcheme CHellas ARoychoudhury APaiva AAbreu RStorey MGama KSiegmund J(2024)Let's Ask AI About Their Programs: Exploring ChatGPT's Answers To Program Comprehension QuestionsProceedings of the 46th International Conference on Software Engineering: Software Engineering Education and Training10.1145/3639474.3640058(221-232)Online publication date: 14-Apr-2024
https://dl.acm.org/doi/10.1145/3639474.3640058
Macneil SDenny PTran ALeinonen JBernstein SHellas ASarsa SKim J(2024)Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language ModelsProceedings of the 26th Australasian Computing Education Conference10.1145/3636243.3636245(11-18)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1145/3636243.3636245
Silva de Oliveira GGao ZHeckman SLynch CStephenson BStone JBattestilli LRebelsky SShoop L(2024)Exploring Novice Programmers' Testing Behavior: A First Step to Define Coding StruggleProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630851(1251-1257)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630851
Mafukidze HNechibvute AYahya ABadruddin IKamangar SHussien M(2024)Development of a Modularized Undergraduate Data Science and Big Data Curricular Using No-Code Software Development ToolsIEEE Access10.1109/ACCESS.2024.342924112(100939-100956)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3429241
Kaleem MHassan MKhurshid S(2024)A Machine Learning-Based Adaptive Feedback System to Enhance Programming Skill Using Computational ThinkingIEEE Access10.1109/ACCESS.2024.339187312(59431-59440)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3391873
Gill SGoolsby BPawluk D(2023)Kinesthetic Feedback for Understanding Program ExecutionSensors10.3390/s2311515923:11(5159)Online publication date: 29-May-2023
https://doi.org/10.3390/s23115159
Haaranen LAhrenberg LHellas A(2023)Decades of Striving for Pedagogical and Technological AlignmentProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631809(1-8)Online publication date: 13-Nov-2023
https://dl.acm.org/doi/10.1145/3631802.3631809
Shin AKazerouni A(2023)A Model of How Students Engineer Test Cases With FeedbackACM Transactions on Computing Education10.1145/362860424:1(1-31)Online publication date: 20-Oct-2023
https://dl.acm.org/doi/10.1145/3628604
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A multi-national, multi-institutional study of assessment of programming skills of first-year CS students

A first-year experience report on a model for statewide deployment of CS principles courses (abstract only)

Insights into Finnish first-year pre-service teachers' twenty-first century skills