research-article

Open access

Efficiency of Learning from Proof Blocks Versus Writing Proofs

Authors:

Benjamin Cosman,

Geoffrey L. HermanAuthors Info & Claims

SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1

Pages 472 - 478

https://doi.org/10.1145/3545945.3569797

Published: 03 March 2023 Publication History

Abstract

Proof Blocks is a software tool that provides students with a scaffolded proof-writing experience, allowing them to drag and drop prewritten proof lines into the correct order instead of starting from scratch. In this paper we describe a randomized controlled trial designed to measure the learning gains of using Proof Blocks for students learning proof by induction. The study participants were 332 students recruited after completing the first month of their discrete mathematics course. Students in the study took a pretest and read lecture notes on proof by induction, completed a brief (less than 1 hour) learning activity, and then returned one week later to complete the posttest. Depending on the experimental condition that each student was assigned to, they either completed only Proof Blocks problems, completed some Proof Blocks problems and some written proofs, or completed only written proofs for their learning activity. We find that students in the early phases of learning about proof by induction are able to learn just as much from reading lecture notes and using Proof Blocks as by reading lecture notes and writing proofs from scratch, but in far less time on task. This finding complements previous findings that Proof Blocks are useful exam questions and are viewed positively by students.

References

[1]

Stacy A. Brown. 2008. Exploring epistemological obstacles to the development of mathematics induction. In The 11th Conference for Research on Undergraduate Mathematics Education. 1--19.

[2]

Stacy A. Brown. 2014. On skepticism and its role in the development of proof in the classroom. Educational Studies in Mathematics, Vol. 86, 3 (July 2014), 311--335. https://doi.org/10.1007/s10649-014-9544-4

[3]

Elizabeth G Creamer. 2017. An introduction to fully integrated mixed methods research. sage publications.

[4]

John W. Creswell. 2007. Designing and conducting mixed methods research. SAGE Publications, Thousand Oaks, Calif. 2006008436

[5]

Paul Denny, Andrew Luxton-Reilly, and Beth Simon. 2008. Evaluating a new exam question: Parsons problems. In Proceedings of the fourth international workshop on computing education research. 113--124.

Digital Library

[6]

Yuemeng Du, Andrew Luxton-Reilly, and Paul Denny. 2020. A Review of Research on Parsons Problems. In Proceedings of the Twenty-Second Australasian Computing Education Conference (ACE'20). Association for Computing Machinery, New York, NY, USA, 195--202. https://doi.org/10.1145/3373165.3373187

Digital Library

[7]

Barbara J Ericson, Paul Denny, James Prather, Rodrigo Duran, Arto Hellas, Juho Leinonen, Craig S Miller, Briana Morrison, Janice L Pearce, and Susan H Rodger. 2022. Planning a Multi-institutional and Multi-national Study of the Effectiveness of Parsons Problems. In Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 2. 576--577.

Digital Library

[8]

Barbara J Ericson, James D Foley, and Jochen Rick. 2018. Evaluating the efficiency and effectiveness of adaptive parsons problems. In Proceedings of the 2018 ACM Conference on International Computing Education Research. 60--68.

Digital Library

[9]

Barbara J Ericson, Lauren E Margulieux, and Jochen Rick. 2017. Solving parsons problems versus fixing and writing code. In Proceedings of the 17th Koli Calling International Conference on Computing Education Research. 20--29.

Digital Library

[10]

Ken Goldman, Paul Gross, Cinda Heeren, Geoffrey Herman, Lisa Kaczmarczyk, Michael C Loui, and Craig Zilles. 2008. Identifying important and difficult concepts in introductory computing courses using a delphi process. In Proceedings of the 39th SIGCSE technical symposium on Computer science education. 256--260.

Digital Library

[11]

Guershon Harel. 2001. The development of mathematical induction as a proof scheme: A model for DNR-based instruction. In In. Citeseer.

[12]

Mark Hodds, Lara Alcock, and Matthew Inglis. 2014. Self-explanation training improves proof comprehension. Journal for Research in Mathematics Education, Vol. 45, 1 (2014), 62--101.

[13]

Hans Niels Jahnke and Ralf Wambach. 2013. Understanding what a proof is: a classroom-based approach. ZDM, Vol. 45, 3 (May 2013), 469--482. https://doi.org/10.1007/s11858-013-0502-x

[14]

Klaus Krippendorff. 2004. Reliability in content analysis: Some common misconceptions and recommendations. Human communication research, Vol. 30, 3 (2004), 411--433.

[15]

Sean Larsen and Michelle Zandieh. 2008. Proofs and refutations in the undergraduate mathematics classroom. Educational Studies in Mathematics, Vol. 67, 3 (March 2008), 205--216. https://doi.org/10.1007/s10649-007-9106-0

[16]

Aliza Malek and Nitsa Movshovitz-Hadar. 2011. The effect of using transparent pseudo-proofs in linear algebra. Research in Mathematics Education, Vol. 13, 1 (2011), 33--58.

[17]

David Miller, Nicole Infante, and Keith Weber. 2018. How mathematicians assign points to student proofs. The Journal of Mathematical Behavior, Vol. 49 (2018), 24--34.

[18]

Dale Parsons and Patricia Haden. 2006. Parson's Programming Puzzles: A Fun and Effective Learning Tool for First Programming Courses. In Proceedings of the 8th Australasian Conference on Computing Education - Volume 52 (Hobart, Australia) (ACE '06). Australian Computer Society, Inc., AUS, 157--163.

Digital Library

[19]

Seth Poulsen, Shubhang Kulkarni, Geoffrey Herman, and Matthew West. 2022a. Efficient Partial Credit Grading of Proof Blocks Problems. https://doi.org/10.48550/ARXIV.2204.04196

[20]

Seth Poulsen, Mahesh Viswanathan, Geoffrey L Herman, and Matthew West. 2021. Evaluating Proof Blocks Problems as Exam Questions. In Proceedings of the 17th ACM Conference on International Computing Education Research. 157--168.

Digital Library

[21]

Seth Poulsen, Mahesh Viswanathan, Geoffrey L. Herman, and Matthew West. 2022b. Proof Blocks: Autogradable Scaffolding Activities for Learning to Write Proofs. In Proceedings of the 2022 ACM Conference on Innovation and Technology in Computer Science Education.

Digital Library

[22]

Gila Ron and Tommy Dreyfus. 2004. The Use of Models in Teaching Proof by Mathematical Induction. International Group for the Psychology of Mathematics Education (2004).

[23]

Somali Roy. 2014. Evaluating novel pedagogy in higher education: a case study of e-proofs. thesis. Loughborough University. /articles/thesis/Evaluating_novel_pedagogy_in_higher_education_a_case_study_of_e-proofs/9374297/1

[24]

Annie Selden and John Selden. 2008. Overcoming Students' Difficulties in Learning to Understand and Construct Proofs. In Making the Connection, Marilyn P. Carlson and Chris Rasmussen (Eds.). The Mathematical Association of America, Washington DC, 95--110. https://doi.org/10.5948/UPO9780883859759.009

[25]

Andreas J Stylianides, Kristen N Bieda, and Francesca Morselli. 2016. Proof and argumentation in mathematics education research. In The second handbook of research on the psychology of mathematics education. Brill Sense, 315--351.

[26]

GJ Stylianides, AJ Stylianides, and K Weber. 2017. Research on the teaching and learning of proof: Taking stock and moving forward. In Compendium for Research in Mathematics Education, Jinfa Cai (Ed.). National Council of Teachers of Mathematics, Chapter 10, 237--266.

[27]

Gabriel J. Stylianides and Andreas J. Stylianides. 2009. Facilitating the Transition from Empirical Arguments to Proof. Journal for Research in Mathematics Education, Vol. 40, 3 (2009), 314--352. https://www.jstor.org/stable/40539339 Publisher: National Council of Teachers of Mathematics.

[28]

Lev Semenovich Vygotsky. 1978. Mind in society: The development of higher psychological processes. Harvard university press.

[29]

Keith Weber. 2001. Student difficulty in constructing proofs: The need for strategic knowledge. Educational Studies in Mathematics, Vol. 48, 1 (Oct. 2001), 101--119. https://doi.org/10.1023/A:1015535614355

[30]

Keith Weber and Lara Alcock. 2004. Semantic and Syntactic Proof Productions. Educational Studies in Mathematics, Vol. 56, 2 (July 2004), 209--234. https://doi.org/10.1023/B:EDUC.0000040410.57253.a1

[31]

Keith Weber, E Fuller, JP Mejia-Ramos, Kristen Lew, Philip Benjamin, and Aron Samkoff. 2012. Do generic proofs improve proof comprehension. In Proceedings of the 15th Annual Conference on Research In Undergraduate Mathematics Education. Citeseer, 480--495.

[32]

Nathaniel Weinman, Armando Fox, and Marti A Hearst. 2021. Improving Instruction of Programming Patterns with Faded Parsons Problems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1--4.

Digital Library

[33]

Matthew West, Geoffrey L. Herman, and Craig Zilles. 2015. PrairieLearn: Mastery-based Online Problem Solving with Adaptive Scoring and Recommendations Driven by Machine Learning. In 2015 ASEE Annual Conference & Exposition. ASEE Conferences, Seattle, Washington, 26.1238.1--26.1238.14. https://peer.asee.org/24575.

[34]

Rand Wilcox. 2019. A robust nonparametric measure of effect size based on an analog of Cohen's d, plus inferences about the median of the typical difference. Journal of Modern Applied Statistical Methods, Vol. 17, 2 (2019), 1.

[35]

Craig Zilles, Matthew West, Geoffrey L Herman, and Timothy Bretl. 2019. Every University Should Have a Computer-Based Testing Facility. In CSEDU (1). 414--420.

Cited By

Smith DPoulsen SEmeka CWu ZHaynes-Magyar CZilles C(2024)Distractors Make You Pay Attention: Investigating the Learning Outcomes of Including Distractor Blocks in Parsons ProblemsProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671114(177-191)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671114
Poulsen SSarsa SPrather JLeinonen JBecker BHellas ADenny PReeves BStephenson BStone JBattestilli LRebelsky SShoop L(2024)Solving Proof Block Problems Using Large Language ModelsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630928(1063-1069)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630928
Poulsen SGertner YChen HCosman BWest MHerman GStephenson BStone JBattestilli LRebelsky SShoop L(2024)Disentangling the Learning Gains from Reading a Book Chapter and Completing Proof Blocks ProblemsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630831(1056-1062)Online publication date: 7-Mar-2024
https://doi.org/10.1145/3626252.3630831
Show More Cited By

Index Terms

Efficiency of Learning from Proof Blocks Versus Writing Proofs

Recommendations

Proof Blocks: Autogradable Scaffolding Activities for Learning to Write Proofs
ITiCSE '22: Proceedings of the 27th ACM Conference on on Innovation and Technology in Computer Science Education Vol. 1

In this software tool paper we present Proof Blocks, a tool which enables students to construct mathematical proofs by dragging and dropping prewritten proof lines into the correct order. We present both implementation details of the tool, as well as a ...
Evaluating Proof Blocks Problems as Exam Questions
ICER 2021: Proceedings of the 17th ACM Conference on International Computing Education Research

Proof Blocks is a novel software tool which enables students to write mathematical proofs by dragging and dropping prewritten lines into the correct order, rather than writing a proof completely from scratch. We used Proof Blocks problems as exam ...
Disentangling the Learning Gains from Reading a Book Chapter and Completing Proof Blocks Problems
SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1

Background : Proof Blocks is a software tool that enables students to construct proofs by assembling prewritten lines and gives them automated feedback. Prior research has shown that students learn as much from an activity where they use Proof Blocks as ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGCSE 2023: Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1

March 2023

1481 pages

ISBN:9781450394314

DOI:10.1145/3545945

General Chairs:
Maureen Doyle
Northern Kentucky University, USA
,
Ben Stephenson
University of Calgary, Canada
,
Program Chairs:
Brian Dorn
University of Nebraska at Omaha, USA
,
Leen-Kiat Soh
University of Nebraska-Lincoln, USA
,
Lina Battestilli
North Carolina State University, USA

Copyright © 2023 Owner/Author.

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.

Sponsors

SIGCSE: ACM Special Interest Group on Computer Science Education

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 03 March 2023

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SIGCSE 2023

Sponsor:

SIGCSE

SIGCSE 2023: The 54th ACM Technical Symposium on Computer Science Education

March 15 - 18, 2023

Toronto ON, Canada

Acceptance Rates

Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

Upcoming Conference

SIGCSE TS 2025

Sponsor:
sigcse

The 56th ACM Technical Symposium on Computer Science Education

February 26 - March 1, 2025

Pittsburgh , PA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
288
Total Downloads

Downloads (Last 12 months)125
Downloads (Last 6 weeks)17

Reflects downloads up to 23 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Smith DPoulsen SEmeka CWu ZHaynes-Magyar CZilles C(2024)Distractors Make You Pay Attention: Investigating the Learning Outcomes of Including Distractor Blocks in Parsons ProblemsProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671114(177-191)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3632620.3671114
Poulsen SSarsa SPrather JLeinonen JBecker BHellas ADenny PReeves BStephenson BStone JBattestilli LRebelsky SShoop L(2024)Solving Proof Block Problems Using Large Language ModelsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630928(1063-1069)Online publication date: 7-Mar-2024
https://dl.acm.org/doi/10.1145/3626252.3630928
Poulsen SGertner YChen HCosman BWest MHerman GStephenson BStone JBattestilli LRebelsky SShoop L(2024)Disentangling the Learning Gains from Reading a Book Chapter and Completing Proof Blocks ProblemsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630831(1056-1062)Online publication date: 7-Mar-2024
https://doi.org/10.1145/3626252.3630831
Poulsen SKulkarni SHerman GWest M(2023)Efficient Feedback and Partial Credit Grading for Proof Blocks ProblemsArtificial Intelligence in Education10.1007/978-3-031-36272-9_41(502-514)Online publication date: 3-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-36272-9_41

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents