skip to main content
research-article

Teaching Introductory Programming: A Quantitative Evaluation of Different Approaches

Published: 29 December 2014 Publication History

Abstract

Teaching programming to beginners is a complex task. In this article, the effects of three factors—choice of programming language, problem-solving training, and the use of formative assessment—on learning to program were investigated. The study adopted an iterative methodological approach carried out across 4 consecutive years. To evaluate the effects of each factor (implemented as a single change in each iteration) on students’ learning performance, the study used quantitative, objective metrics. The findings revealed that using a syntactically simple language (Python) instead of a more complex one (Java) facilitated students’ learning of programming concepts. Moreover, teaching problem solving before programming yielded significant improvements in student performance. These two factors were found to have variable effects on the acquisition of basic programming concepts. Finally, it was observed that effective formative feedback in the context of introductory programming depends on multiple parameters. The article discusses the implications of these findings, identifies avenues for further research, and argues for the importance of studies in computer science education anchored on sound research methodologies to produce generalizable results.

References

[1]
A. Agresti and I. M. Liu. 1999. Modeling a categorical variable allowing arbitrarily many category choices. Biometrics, 55, 3 (Sept. 1999), 936--943.
[2]
V. L. Almstrum, O. Hazzan, M. Guzdial, and M. Petre. 2005, Challenges to computer science education research. ACM SIGCSE Bull. 37, 1 (2005), 191--192.
[3]
T. Beaubouef and J. Mason. 2005. Why the high attrition rate for computer science students: Some thoughts and observations. ACM SIGCSE Bull. 37, 2 (2005), 103--106.
[4]
L. Beck and A. Chizhik. 2013. Cooperative learning instructional methods for CS1: Design, implementation, and evaluation. ACM Trans. Comput. Educ. (TOCE) 13, 3 (Aug. 2013), Article 10.
[5]
J. Bennedsen and M. E. Caspersen. 2007. Assessing process and product: A practical lab exam for an introductory programming course 1. Inn. Teaching Learning Inf. Comput. Sci. 6, 4 (Oct. 2007), 183--202.
[6]
S. B. Berenson, K. M. Slaten, L. Williams, and C. W. Ho. 2004. Voices of women in a software engineering course: Reflections on collaboration. J. Educ. Res. Comput. (JERIC), 4, 1 (March 2004), Article 3.
[7]
P. Black. 1999. Assessment, learning theories and testing systems. In Learners, Learning and Assessment. Paul Chapman Publishing, London, 118--134.
[8]
P. Black and D. Wiliam. 1998. Assessment and classroom learning. Assess. Educ. 5, 1 (1998), 7--74.
[9]
S. J. Bostock. 2004. Motivation and electronic assessment. Effective Learning and Teaching in Computing. Routledge Falmer, London. 86--99.
[10]
G. Braught, T. Wahls, and L. M. Eby. 2011. The case for pair programming in the computer science classroom. ACM Trans. Comput. Educ. (TOCE), 11, 1, Article 2 (Feb. 2011).
[11]
A. L. Brown. 1992. Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. J. Learn. Sci. 2, 2 (1992), 141--178.
[12]
M. E. Califf and M. Goodwin. 2002. Testing skills and knowledge: Introducing a laboratory exam in CS1. ACM SIGCSE Bull. 34, 1 (Feb. 2002), 217--221.
[13]
J. Carter and R. Boyle. 2002. Teaching delivery issues: Lessons from computer science. J. Inf. Technol. 1, 2 (2002), 65--90.
[14]
A. T. Chamillard and K. A. Braun. 2000. Evaluating programming ability in an introductory computer science course. ACM SIGCSE Bull. 32, 1 (March 2000), 212--216.
[15]
A. Collins. 1992. Toward a design science of education. Springer, Berlin, 15--22.
[16]
A. Collins, D. Joseph, and K. Bielaczyc. 2004. Design research: Theoretical and methodological issues. J. Learn. Sci. 13, 1 (2004), 15--42.
[17]
Computing Curricula. 2001. IEEE CS, ACM Joint Task Force on Computing Curricula. IEEE Computer Society Press and ACM Press. Retrieved from http://www.acm.org/education/curricula.html.
[18]
Computing Curricula: The overview report. 2005. IEEE CS, ACM Joint Task Force on Computing Curricula. IEEE Computer Society Press and ACM Press. Retrieved from http://www.acm.org/education/curricula.html.
[19]
A. T. Corbett and J. R. Anderson. 1990. The effect of feedback control on learning to program with the LISP tutor. In Proceedings of the Twelfth Annual Conference of the Cognitive Science Society. 796--803.
[20]
A. T. Corbett and J. R. Anderson. 2001. Locus of feedback control in computer-based tutoring: Impact on learning rate, achievement and attitudes. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 245--252.
[21]
C. Daly and J. Waldron. 2004. Assessing the assessment of programming ability. ACM SIGCSE Bull. 36, 1 (March 2004), 210--213.
[22]
DARPA. 2010. Computer Science—Science, Technology, Engineering, and Mathematics (CS-STEM) Education Research Announcement (RA). DARPA-RA-10-03.
[23]
P. Denny, A. Luxton-Reilly, and E. Tempero. 2012. All syntax errors are not equal. In Proceedings of the 17th ACM Annual Conference on Innovation and Technology in Computer Science Education (ITiCSE’12). 75--80.
[24]
P. Denny, A. Luxton-Reilly, E. Tempero, and J. Hendrickx. 2011. Understanding the syntax barrier for novices. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education (ITiCSE’11). ACM, New York, 208--212.
[25]
N. K. Denzin and Y. S. Lincoln, (Eds.). 1998. The Landscape of Qualitative Research: Theories and Issues. Sage, Thousand Oaks, CA.
[26]
B. Du Boulay. 1986. Some Difficulties of Learning to Program. J. Educ. Comput. Res. 2, 1 (1986), 57--73.
[27]
K. Ecclestone. 1998. “Just tell me what to do” barriers to assessment-in-learning in higher education. In Scottish Educational Research Association Annual Conference, University of Dundee. 25--26.
[28]
A. Ehlert and C. Schulte. 2009. Empirical comparison of objects-first and objects-later. In Proceedings of the 5th International Workshop on Computing Education Research Workshop. ACM Press, New York, NY, 15--26.
[29]
M. H. Goldwasser and D. Letscher. 2008. Using Python to Teach Object-Oriented Programming in CS1. Innov. Technol. Comput. Sci. Ed. (June 2008), 30--32.
[30]
L. Grandell, M. Peltomäki, R. J. Back, and T. Salakoski. 2006. Why complicate things?: Introducing programming in high school using Python. In Proceedings of the 8th Australasian Conference on Computing Education—Volume 52. Australian Computer Society, Inc., 71--80.
[31]
M. Guzdial. 2009. Education: Teaching computing to everyone. Commun. ACM, 52, 5 (2009), 31--33.
[32]
R. Higgins, P. Hartley, and A. Skelton. 2002. The conscientious consumer: Reconsidering the role of assessment feedback in student learning. Studies Higher Educ. 27, 1 (2002), 53--64.
[33]
A. Irons. 2010. An Investigation into the Impact of Formative Feedback on the Student Learning Experience Doctoral dissertation. Durham University, Durham, UK.
[34]
M. C. Jadud. 2005. A first look at novice compilation behaviour using BlueJ. Comput. Sci. Educ. 15, 1 (2005), 25--40.
[35]
J. Kasurinen and U. Nikula. 2007. Lower dropout rates and better grades through revised course infrastructure. In Proceedings of the 10th IASTED International Conference on Computers and Advanced Technology in Education. ACTA Press, Calgary, Canada. 152--157.
[36]
M. Kölling. 1999. The problem of teaching object-oriented programming, Part 1: Languages. J. Obj. Orient. Prog. 11, 8, 8--15.
[37]
V. Koshy. 2009. Action Research for Improving Educational Practice: A Step-by-Step Guide. Sage, Thousand Oaks, CA.
[38]
S. K. Kummerfeld and J. Kay. 2003. The neglected battle fields of syntax errors. In Proceedings of the Fifth Australasian Conference on Computing Education—Vol. 20. Australian Computer Society, Inc., 105--111.
[39]
C. B. Lee. 2013. Experience report: CS1 in MATLAB for non-majors, with media computation and peer instruction. In Proceedings of the 44th ACM Technical Symposium on Computer Science Education. ACM Press, New York, NY, 35--40.
[40]
M. Lilley and T. Barker. 2007. Students’ perceived usefulness of formative feedback for a computer-adaptive test. Electron. J. e-learning 5 (2007), 31--38.
[41]
T. Lillis and J. Turner. 2001. Student writing in higher education: Contemporary confusion, traditional concerns. Teaching Higher Educ. 6, 1 (2001), 57--68.
[42]
M. C. Linn and J. Dalbey. 1989. Cognitive consequences of programming instruction. In Studying the Novice Programmer, E. Soloway and J. C. Spohrer (Eds.). Lawrence Erlbaum Associates, Hillsdale, NJ, 57--81.
[43]
R. Lister, E. S. Adams, S. Fitzgerald, W. Fone, J. Hamer, M. Lindholm, and L. Thomas. 2004. A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bull. 36, 4 (2007), 119--150.
[44]
L. Ma, J. Ferguson, M. Roper, and M. Wood. 2007. Investigating the viability of mental models held by novice programmers. ACM SIGCSE Bull. 39, 1 (March 2007), 499--503.
[45]
L. Mannila and M. de Raadt. 2006. An objective comparison of languages for teaching introductory programming. In Proceedings of the 6th Baltic Sea Conference on Computing Education Research: Koli Calling 2006. ACM, New York, 32--37.
[46]
L. Mannila, M. Peltomki, and T. Salakoski. 2006. What about a simple language? Analyzing the difficulties in learning to program. Comput. Sci. Educ. 16, 3 (2006), 211--227.
[47]
M. McCracken, V. Almstrum, D. Diaz, M. Guzdial, D. Hagan, Kolikant, C. Laxer, L. Thomas, I. Utting, and T. Wilusz. 2001. A multi-national, multi-institutional study of assessment of programming skills of first-year CS students. ACM SIGCSE Bull. 33, 4 (Dec. 2001), 125--180.
[48]
C. McDowell, L. Werner, H. E. Bullock, and J. Fernald. 2003. The impact of pair programming on student performance, perception and persistence. In Proceedings of the 25th International Conference on Software Engineering. IEEE Computer Society, 602--607.
[49]
A. McGettrick, R. Boyle, R. Ibbett, J. Lloyd, G. Lovegrove, and K. Mander. 2004. Grand challenges in computing: Education. British Computer Society.
[50]
B. N. Miller and D. L. Ranum. 2005. Teaching an introductory computer science sequence with Python. In Proceedings of the 38th Midwest Instructional and Computing Symposium.
[51]
B. Miller and D. Ranum. 2006. Freedom to succeed: A three course introductory sequence using Python and Java. J. Comput. Sci. Colleges 22, 1 (2006), 106--116.
[52]
M. Molina, E. Castro, and E. Castro. 2007. Teaching experiments within design research. Int. J. Interdisc. Social Soc. Sci. 2, 4 (2007), 435--440.
[53]
R. Moreno. 2004. Decreasing cognitive load for novice students: Effects of explanatory versus corrective feedback in discovery-based multimedia. Instruct. Sci. 32, 1--2 (2004), 99--113.
[54]
National Audit Office. Staying the course: The retention of students in higher education. 2007. Report by the National Audit Office, 44. Retrieved from http://www.nao.org.uk/report/staying-the-course-the-retention-of-students-in-higher-education/.
[55]
U. Nikula, O. Gotel, and J. Kasurinen. 2011. A motivation guided holistic rehabilitation of the first programming course. ACM Trans. Comput. Educ. (TOCE) 11, 4 (2011), 24.
[56]
J. D. Oldham. 2005. What happens after Python in CS1? J. Comput. Sci. Colleges 20, 6 (2005), 7--13.
[57]
D. B. Palumbo. 1990. Programming language/problem-solving research: A review of relevant issues. Rev. Educ. Res. 60, 1 (1990), 65--89.
[58]
H. Patterson-McNeill. 2006. Experience: From C++ to Python in 3 easy steps. J. Comput. Sci. Colleges 22, 2 (2006), 92--96.
[59]
A. Pears and L. Malmi. 2009. Values and objectives in computing education research. ACM Trans. Comput. Educ. (TOCE), 9, 3 (2009), 15.
[60]
A. Pears, S. Seidman, L. Malmi, L. Mannila, E. Adams, J. Bennedsen, M. Devlin, and J. Paterson. 2007. A survey of literature on the teaching of introductory programming. ACM SIGCSE Bull. 39, 4 (Dec. 2007), 204--223.
[61]
G. Polya. 1973. How to Solve It: A New Aspect of Mathematical Method. Princeton University Press.
[62]
L. Porter, C. Bailey Lee, B. Simon, Q. Cutts, and D. Zingaro. 2011. Experience report: A multi-classroom report on the value of peer instruction. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education. ACM, New York, 138--142.
[63]
Quality Assurance Agency for Higher Education. 2003. Learning from subject review. Retrieved from http://www.qaa.ac.uk/Publications/InformationAndGuidance/Documents/learningFromSubjectReview.pdf.
[64]
R Development Core Team. 2010. R: A language and environment for statistical computing. R Foundation Statistical Computing.
[65]
A. Radenski. 2006. Python First: A lab-based digital introduction to computer science. ACM SIGCSE Bull. 38, 3 (2006), 197--201.
[66]
J. Randolph, G. Julnes, E. Sutinen, and S. Lehman. 2008. A methodological review of computer science education research. J. Inf. Tech. Educ.: Res. 7, 1 (2008), 135--162.
[67]
T. Reeves. 2011. Can educational research be both rigorous and relevant. Educ. Des. 1, 4 (2011), 1--24.
[68]
V. G. Renumol, D. Janakiram, and S. Jayaprakash. 2010. Identification of cognitive processes of effective and ineffective students during computer programming. ACM Trans. Comput. Educ. (TOCE) 10. 3 (2010), 10.
[69]
A. Robins, J. Rountree, and N. Rountree. 2003. Learning and teaching programming: A review and discussion. Comput. Sci. Educ. 13, 2 (2003), 137--172.
[70]
J. Rogalski and R. Samurçay. 1990. Acquisition of programming knowledge and skills. Psych. Prog. 18 (1990), 157--174.
[71]
M. B. Rosson, J. M. Carroll, and H. Sinha. 2011. Orientation of undergraduates toward careers in the computer and information sciences: Gender, self-efficacy and social support. ACM Trans. Comput. Educ. (TOCE) 11, 3 (2011), 14.
[72]
C. Rust. 2002. The impact of assessment on student learning: How can the research literature practically help to inform the development of departmental assessment strategies and learner-centred assessment practices? Active Learn. Higher Educ. 3, 2 (July 2002), 145--158.
[73]
C. Shannon. 2003. Another breadth-first approach to CS I using python. ACM SIGCSE Bull. 35, 1 (Sept. 2003), 248--251.
[74]
V. J. Shute. 2008. Focus on formative feedback. Rev. Educ. Res. 78, 1 (2008), 153--189.
[75]
R. M. Siegfried, D. Chays, and K. G. Herbert. 2008. Will there ever be consensus on cs1. In Proceedings of the 2008 International Conference on Frontiers in Education: Computer Science and Computer Engineering--FECS. Vol. 8, 18--23.
[76]
B. Simon, M. Kohanfars, J. Lee, K. Tamayo, and Q. Cutts. 2010. Experience report: Peer instruction in introductory computing. In Proceedings of the 41st ACM Technical Symposium on Computer Science Education. ACM, New York, 341--345.
[77]
R. E. Slavin. 2003. A reader's guide to scientifically based research. Educ. Leadership 60, 5 (2003), 12--16.
[78]
R. H. Sloan and P. Troy. 2008. CS 0.5: A better approach to introductory computer science for majors. ACM SIGCSE Bull. 40, 1 (2008), 271--275.
[79]
D. Sleeman, R. T. Putnam, J. Baxter, and L. Kuspa. 1988. An introductory Pascal class: A case study of students' errors. Teaching and Learning Computer Programming: Multiple Research Perspectives. RE Mayer (Ed.). Lawrence Erlbaum Associates, Hillsdale, NJ, 237--257.
[80]
J. Stamey and S. Sheel. 2010. A boot camp approach to learning programming in a CS0 course. J. Comput. Sci. Colleges 25, 5 (2010), 34--40.
[81]
A. Stefik and S. Siebert. 2013. An empirical investigation into programming language syntax. ACM Trans. Comput. Educ. (TOCE) 13, 4 (2013), 19.
[82]
L. Stenhouse. 1975. An Introduction to Curriculum Research and Development. Heinemann, London.
[83]
J. O. Talton, D. L. Peterson, S. Kamin, D. Israel, and J. Al-Muhtadi. 2006. Scavenger hunt: Computer science retention through orientation. ACM SIGCSE Bull. 38, 1 (2006), 443--447.
[84]
J. J. Tu and J. R. Johnson. 1990. Can computer programming improve problem-solving ability? ACM SIGCSE Bulletin 22, 2 (1990), 30--33.
[85]
S. Turner and G. Hill. 2007. Robots in problem-solving and programming. In Proceedings of the 8th Annual Conference of the Subject Centre for Information and Computer Sciences.
[86]
U.S. Department of Education. 2001. No Child Left Behind Act. Retrieved from http://www2.ed.gov/policy/elsec/leg/esea02/index.html.
[87]
M. R. Weaver. 2006. Do students value feedback? Student perceptions of tutors’ written responses. Assess. Eval. Higher Educ. 31, 3 (2006), 379--394.
[88]
J. M. Zelle. 1999. Python as a first language. In Proceedings of 13th Annual Midwest Computer Conference. Vol. 2.

Cited By

View all

Index Terms

  1. Teaching Introductory Programming: A Quantitative Evaluation of Different Approaches

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Transactions on Computing Education
      ACM Transactions on Computing Education  Volume 14, Issue 4
      February 2015
      116 pages
      EISSN:1946-6226
      DOI:10.1145/2698235
      Issue’s Table of Contents
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 29 December 2014
      Accepted: 01 August 2014
      Revised: 01 July 2014
      Received: 01 July 2013
      Published in TOCE Volume 14, Issue 4

      Check for updates

      Author Tags

      1. CS1
      2. Empirical studies
      3. formative feedback
      4. learning programming
      5. novice programmers
      6. problem solving
      7. programming languages
      8. teaching strategies

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)314
      • Downloads (Last 6 weeks)27
      Reflects downloads up to 25 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all

      View Options

      Login options

      Full Access

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media