research-article

Impact of Hint Content on Performance and Learning: A Study with Primary School Children in a Scratch Course

Authors:

Luisa Greifenstein,

Gordon FraserAuthors Info & Claims

WiPSCE '23: Proceedings of the 18th WiPSCE Conference on Primary and Secondary Computing Education Research

Article No.: 7, Pages 1 - 10

https://doi.org/10.1145/3605468.3605498

Published: 27 September 2023 Publication History

Abstract

The implementation of computational thinking concepts in primary school curricula usually includes programming activities. As primary school teachers often lack subject knowledge, they may struggle to help students during these programming activities. Additional support can be provided by automated program analysis, for example in terms of hints on conceptual knowledge related to bad coding patterns observed, or procedural hints on how to solve the task at hand. However, care has to be taken since these hints need to balance (1) helping students to perform a specific task successfully, while nevertheless (2) ensuring a learning effect beyond the specific task. To understand the effects of different types of hints we therefore conducted a study with 36 children aged 7–12 in 10 programming courses. After being introduced to basic programming structures in three units, the children were tasked to debug six Scratch programs using different types of hints, where we observed that procedural hints have the strongest impact on performance. In order to examine an impact on the transfer of learned knowledge, we observed the children’s difficulties during the successive fifth unit, in which they created their own projects. The results of the fifth unit show that having received a procedural hint on a specific pattern during the fourth unit leads to slightly fewer bad related code patterns but also to slightly fewer good code patterns. Considering these results together with the subjective perceptions of the children, we can derive insights into how to best support performance and learning using (automated) feedback.

References

[1]

Martina Benvenuti, Sara Giovagnoli, Elvis Mazzoni, 2019. Using educational robot to enhance the potential of creative thinking in children. In 1st Symposium on Psychology-Based Technologies. 1–10.

[2]

Alexander Best, Christian Borowski, Kathrin Büttner, Rita Freudenberg, Martin Fricke, Kathrin Haselmeier, Henry Herper, Volkmar Hinz, Ludger Humbert, Dorothee Müller, 2019. Kompetenzen für informatische Bildung im Primarbereich. LOG IN 38, 1 (2019), 1–36.

[3]

Laura Caspari, Luisa Greifenstein, Ute Heuer, and Gordon Fraser. 2023. ScratchLog: Live Learning Analytics for Scratch. In Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1. 403–409.

Digital Library

[4]

Dave Catlin, Martin Kandlhofer, and Stephanie Holmquist. 2018. EduRobot taxonomy: a provisional schema for classifying educational robots. In International Conference on Robotics in Education 2018.

[5]

Morgane Chevalier, Christian Giang, Laila El-Hamamsy, Evgeniia Bonnet, Vaios Papaspyros, Jean-Philippe Pellet, Catherine Audrin, Margarida Romero, Bernard Baumberger, and Francesco Mondada. 2022. The role of feedback and guidance as intervention methods to foster computational thinking in educational robotics learning activities for primary school. Computers & Education 180 (2022), 104431.

Digital Library

[6]

Daniel Amo Filvà, Marc Alier Forment, Francisco José García-Peñalvo, David Fonseca Escudero, and María José Casañ. 2019. Clickstream for learning analytics to assess students’ behavior with Scratch. Future Generation Computer Systems 93 (2019), 673–686.

Digital Library

[7]

Ayelet Fishbach, Tal Eyal, and Stacey R Finkelstein. 2010. How positive and negative feedback motivate goal pursuit. Social and Personality Psychology Compass 4, 8 (2010), 517–530.

[8]

Ayelet Fishbach, Tal Eyal, and Stacey R Finkelstein. 2010. How positive and negative feedback motivate goal pursuit. Social and Personality Psychology Compass 4, 8 (2010), 517–530.

[9]

Christoph Frädrich, Florian Obermüller, Nina Körber, Ute Heuer, and Gordon Fraser. 2020. Common bugs in scratch programs. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education. 89–95.

Digital Library

[10]

Gordon Fraser, Ute Heuer, Nina Körber, Florian Obermüller, and Ewald Wasmeier. 2021. LitterBox: A Linter for Scratch Programs. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET). 183–188.

Digital Library

[11]

Katharina Geldreich, Mike Talbot, and Peter Hubwieser. 2019. Aufgabe ist nicht gleich Aufgabe–Vielfältige Aufgabentypen bewusst in Scratch einsetzen. Informatik für alle (2019).

[12]

Luisa Greifenstein, Isabella Graßl, and Gordon Fraser. 2021. Challenging but Full of Opportunities: Teachers’ Perspectives on Programming in Primary Schools. In 21st Koli Calling International Conference on Computing Education Research. 1–10.

[13]

Luisa Greifenstein, Florian Obermüller, Ewald Wasmeier, Ute Heuer, and Gordon Fraser. 2021. Effects of Hints on Debugging Scratch Programs: An Empirical Study with Primary School Teachers in Training. In The 16th Workshop in Primary and Secondary Computing Education. 1–10.

[14]

Shuchi Grover, Satabdi Basu, Marie Bienkowski, Michael Eagle, Nicholas Diana, and John Stamper. 2017. A framework for using hypothesis-driven approaches to support data-driven learning analytics in measuring computational thinking in block-based programming environments. ACM Transactions on Computing Education (TOCE) 17, 3 (2017), 1–25.

Digital Library

[15]

Brian Harvey, Daniel D Garcia, Tiffany Barnes, Nathaniel Titterton, Daniel Armendariz, Luke Segars, Eugene Lemon, Sean Morris, and Josh Paley. 2013. Snap!(build your own blocks). In Proceeding of the 44th ACM technical symposium on Computer science education. 759–759.

[16]

John Hattie and Helen Timperley. 2007. The power of feedback. Review of educational research 77, 1 (2007), 81–112.

[17]

Fredrik Heintz, Linda Mannila, and Tommy Färnqvist. 2016. A review of models for introducing computational thinking, computer science and computing in K-12 education. In FIE ’16. 1–9.

[18]

Ludger Humbert and Hermann Puhlmann. 2004. Essential Ingredients of Literacy in Informatics.Informatics and Student Assessment 65 (2004), 76.

[19]

David E Johnson. 2016. Itch: Individual testing of computer homework for scratch assignments. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education. 223–227.

Digital Library

[20]

Max Kesselbacher and Andreas Bollin. 2019. Discriminating programming strategies in scratch: Making the difference between novice and experienced programmers. In Proceedings of the 14th Workshop in Primary and Secondary Computing Education. 1–10.

Digital Library

[21]

Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2018. A systematic literature review of automated feedback generation for programming exercises. ACM Transactions on Computing Education (TOCE) 19, 1 (2018), 1–43.

Digital Library

[22]

Laura R Larke. 2019. Agentic neglect: Teachers as gatekeepers of England’s national computing curriculum. BJET 50, 3 (2019), 1137–1150.

[23]

Irene Lee, Fred Martin, Jill Denner, Bob Coulter, Walter Allan, Jeri Erickson, Joyce Malyn-Smith, and Linda Werner. 2011. Computational thinking for youth in practice. Acm Inroads 2, 1 (2011), 32–37.

Digital Library

[24]

John Maloney, Mitchel Resnick, Natalie Rusk, Brian Silverman, and Evelyn Eastmond. 2010. The scratch programming language and environment. ACM Transactions on Computing Education (TOCE) 10, 4 (2010), 1–15.

Digital Library

[25]

Samiha Marwan, Anay Dombe, and Thomas W Price. 2020. Unproductive help-seeking in programming: What it is and how to address it. In Proceedings of the 2020 ACM conference on innovation and technology in computer science education. 54–60.

Digital Library

[26]

Samiha Marwan, Joseph Jay Williams, and Thomas Price. 2019. An evaluation of the impact of automated programming hints on performance and learning. In Proceedings of the 2019 ACM Conference on International Computing Education Research. 61–70.

Digital Library

[27]

Samiha Marwan, Nicholas Lytle, Joseph Jay Williams, and Thomas Price. 2019. The impact of adding textual explanations to next-step hints in a novice programming environment. In Proceedings of the 2019 ACM conference on innovation and technology in computer science education. 520–526.

Digital Library

[28]

Tilman Michaeli and Ralf Romeike. 2019. Current status and perspectives of debugging in the k12 classroom: A qualitative study. In 2019 ieee global engineering education conference (educon). IEEE, 1030–1038.

[29]

Tilman Michaeli and Ralf Romeike. 2019. Improving debugging skills in the classroom: The effects of teaching a systematic debugging process. In Proceedings of the 14th workshop in primary and secondary computing education. 1–7.

Digital Library

[30]

Maria Montessori. 1959. The absorbent mind. Lulu. com.

[31]

Jesús Moreno-León, Gregorio Robles, and Marcos Román-González. 2015. Dr. Scratch: Automatic Analysis of Scratch Projects to Assess and Foster Computational Thinking. RED-Revista de Educación a Distancia (09 2015).

[32]

Susanne Narciss. 2013. Designing and evaluating tutoring feedback strategies for digital learning. Digital Education Review23 (2013), 7–26.

[33]

Christin Nenner and Nadine Bergner. 2022. Informatics Education in German Primary School Curricula. In Informatics in Schools. A Step Beyond Digital Education: 15th International Conference on Informatics in Schools: Situation, Evolution, and Perspectives, ISSEP 2022, Vienna, Austria, September 26–28, 2022, Proceedings. Springer, 3–14.

Digital Library

[34]

Florian Obermüller, Lena Bloch, Luisa Greifenstein, Ute Heuer, and Gordon Fraser. 2021. Code Perfumes: Reporting Good Code to Encourage Learners. In The 16th Workshop in Primary and Secondary Computing Education. 1–10.

Digital Library

[35]

Thomas W Price, Yihuan Dong, and Dragan Lipovac. 2017. iSnap: towards intelligent tutoring in novice programming environments. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on computer science education. 483–488.

Digital Library

[36]

Thomas W Price, Rui Zhi, and Tiffany Barnes. 2017. Hint generation under uncertainty: The effect of hint quality on help-seeking behavior. In Artificial Intelligence in Education: 18th International Conference, AIED 2017, Wuhan, China, June 28–July 1, 2017, Proceedings 18. Springer, 311–322.

[37]

Jean Salac, Cathy Thomas, Chloe Butler, Ashley Sanchez, and Diana Franklin. 2020. TIPP&SEE: a learning strategy to guide students through use-modify Scratch activities. In Proceedings of the 51st ACM Technical Symposium on Computer Science Education. 79–85.

Digital Library

[38]

Sue Sentance and Andrew Csizmadia. 2017. Computing in the curriculum: Challenges and strategies from a teacher’s perspective. Education and Information Technologies 22, 2 (2017), 469–495.

Digital Library

[39]

Sue Sentance, Jane Waite, and Maria Kallia. 2019. Teachers’ experiences of using primm to teach programming in school. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. 476–482.

Digital Library

[40]

Andreas Stahlbauer, Marvin Kreis, and Gordon Fraser. 2019. Testing scratch programs automatically. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. 165–175.

Digital Library

[41]

Mike Talbot, Katharina Geldreich, Julia Sommer, and Peter Hubwieser. 2020. Re-use of programming patterns or problem solving? representation of scratch programs by TGraphs to support static code analysis. In Proceedings of the 15th Workshop on Primary and Secondary Computing Education. 1–10.

Digital Library

[42]

Giovanni Maria Troiano, Sam Snodgrass, Erinç Argımak, Gregorio Robles, Gillian Smith, Michael Cassidy, Eli Tucker-Raymond, Gillian Puttick, and Casper Harteveld. 2019. Is my game OK Dr. Scratch? Exploring programming and computational thinking development via metrics in student-designed serious games for STEM. In Proceedings of the 18th ACM international conference on interaction design and children. 208–219.

Digital Library

[43]

Benedikt Wisniewski, Klaus Zierer, and John Hattie. 2020. The power of feedback revisited: A meta-analysis of educational feedback research. Frontiers in Psychology 10 (2020), 3087.

[44]

T Wolff, L Hellmig, and A Martens. 2020. STATE OF THE ART IN CURRICULUM RESEARCH FROM THE PERSPECTIVE OF GERMAN COMPUTER SCIENCE TEACHERS. ICERI2020 Proceedings (2020), 9177–9184.

[45]

Aman Yadav, Sarah Gretter, Susanne Hambrusch, and Phil Sands. 2016. Expanding computer science education in schools: understanding teacher experiences and challenges. Computer Science Education 26, 4 (2016), 235–254.

Cited By

Sun CYang SBecker B(2024)Debugging in Computational Thinking: A Meta-analysis on the Effects of Interventions on Debugging SkillsJournal of Educational Computing Research10.1177/0735633124122779362:4(1087-1121)Online publication date: 20-Jan-2024
https://doi.org/10.1177/07356331241227793
Greifenstein LHeuer UFraser GStephenson BStone JBattestilli LRebelsky SShoop L(2024)Hint Cards for Common Ozobot Robot Issues: Supporting Feedback for Learning Programming in Elementary SchoolsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630868(408-414)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630868
Jha NBhowmik PBhagat K(2024)Investigating the effect of multiple try-feedback on students computational thinking skills through online inquiry-based learning platformEducational technology research and development10.1007/s11423-024-10397-372:6(3295-3347)Online publication date: 1-Jul-2024
https://doi.org/10.1007/s11423-024-10397-3

Index Terms

Impact of Hint Content on Performance and Learning: A Study with Primary School Children in a Scratch Course
1. Social and professional topics
  1. Professional topics
    1. Computing education
      1. Computing education programs
        Software engineering education
      2. K-12 education
2. Software and its engineering
  1. Software notations and tools
    1. Context specific languages
      1. Visual languages

Recommendations

Gender Differences in Scratch Programs of Primary School Children
WiPSCE '17: Proceedings of the 12th Workshop on Primary and Secondary Computing Education

It is commonly believed that the attitudes towards programming are strongly dependent on gender among adolescents or adults. Regarding younger children, these differences in the attitudes might be less relevant. To investigate this, we have analyzed the ...
Students perceptions of programming in primary school
WiPSCE '19: Proceedings of the 14th Workshop in Primary and Secondary Computing Education

Since autumn 2018, teachers throughout Sweden are obliged to relate to programming in one way or another in the teaching, especially in the subject of mathematics and technology education. Although teachers should formally work with programming teaching ...
Promoting computational thinking and creativeness in primary school children
TEEM 2017: Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality

This paper presents the preliminary results of the project "Computational Thinking for children education", aimed at promoting computational thinking, creativity and learning amongst primary school children. The didactic activities of the project focus ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

WiPSCE '23: Proceedings of the 18th WiPSCE Conference on Primary and Secondary Computing Education Research

September 2023

173 pages

ISBN:9798400708510

DOI:10.1145/3605468

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 September 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

01JA2021

Conference

WiPSCE '23

WiPSCE '23: The 18th WiPSCE Conference on Primary and Secondary Computing Education Research

September 27 - 29, 2023

Cambridge, United Kingdom

Acceptance Rates

Overall Acceptance Rate 104 of 279 submissions, 37%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
106
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)7

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Sun CYang SBecker B(2024)Debugging in Computational Thinking: A Meta-analysis on the Effects of Interventions on Debugging SkillsJournal of Educational Computing Research10.1177/0735633124122779362:4(1087-1121)Online publication date: 20-Jan-2024
https://doi.org/10.1177/07356331241227793
Greifenstein LHeuer UFraser GStephenson BStone JBattestilli LRebelsky SShoop L(2024)Hint Cards for Common Ozobot Robot Issues: Supporting Feedback for Learning Programming in Elementary SchoolsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630868(408-414)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630868
Jha NBhowmik PBhagat K(2024)Investigating the effect of multiple try-feedback on students computational thinking skills through online inquiry-based learning platformEducational technology research and development10.1007/s11423-024-10397-372:6(3295-3347)Online publication date: 1-Jul-2024
https://doi.org/10.1007/s11423-024-10397-3

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents