skip to main content
10.1145/3564721.3564740acmotherconferencesArticle/Chapter ViewAbstractPublication Pageskoli-callingConference Proceedingsconference-collections
research-article

Exploring the Differences in Students’ Behavioral Engagement With Quizzes and Its Impact on their Performance in a Flipped CS1 Course

Published: 17 November 2022 Publication History

Abstract

There is an increase in the adoption of flipped classroom pedagogy for introductory programming (CS1) courses. In a flipped course, students watch the content videos and complete an accountability quiz before the class, then do active learning activities during the class. The role of students’ behavioral engagement and its impact on learning outcomes is widely studied in education, but little is known about its effect in flipped CS1 courses. This paper analyzes factors related to students’ behavioral engagement with quizzes, such as how much time they spend on quizzes, when they choose to submit the quizzes, and how consistently they space their weekly quiz submissions over a fifteen-week semester. Firstly, group differences based on GPA, gender and prior programming experience (PPE) are explored to understand how behavioral engagement varies among different student populations. Secondly, we analyze the association of behavioral engagement with students’ performance using exam averages. We find that behavioral metrics do not vary based on GPA, PPE, and gender. Further, we find that while the time taken on quizzes and weekly consistency is not correlated with students’ performance, students who submit the quizzes earlier tend to have statistically higher exam averages than those who complete them near the deadlines. These results align with earlier findings and will help instructors understand students’ behavioral approaches to flipped CS1 courses, which can help them tailor their instructions accordingly.

Supplementary Material

MP4 File (COP2271-KoliCalling2022Presentation-v001.mp4)
Presentation Video for "Exploring the Differences in Students? Behavioral Engagement With Quizzes and Its Impact on their Performance in a Flipped CS1 Course" presented at Koli Calling - 2022.

References

[1]
Dan Ariely and Klaus Wertenbroch. 2002. Procrastination, deadlines, and performance: Self-control by precommitment. Psychological science 13, 3 (2002), 219–224.
[2]
Jens Bennedsen and Michael E. Caspersen. 2007. Failure Rates in Introductory Programming. SIGCSE Bull. 39, 2 (June 2007), 32–36. https://doi.org/10.1145/1272848.1272879
[3]
Jacob Bishop and Matthew A Verleger. 2013. The Flipped Classroom: A Survey of the Research. In 2013 ASEE Annual Conference & Exposition. ASEE Conferences, Atlanta, Georgia, 1–18. https://peer.asee.org/22585
[4]
Juris Borzovs, Laila Niedrite, and Darja Solodovnikova. 2015. Factor Affecting Attrition among First Year Computer Science Students: the Case of University of Latvia, In Proceedings of the International Scientific and Practical Conference. Environment. Technology. Resources 3. https://doi.org/10.17770/etr2015vol3.174
[5]
Ricardo Caceffo, Guilherme Gama, and Rodolfo Azevedo. 2018. Exploring Active Learning Approaches to Computer Science Classes. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 922–927. https://doi.org/10.1145/3159450.3159585
[6]
Adam S. Carter, Christopher D. Hundhausen, and Olusola Adesope. 2015. The Normalized Programming State Model: Predicting Student Performance in Computing Courses Based on Programming Behavior. In Proceedings of the Eleventh Annual International Conference on International Computing Education Research (Omaha, Nebraska, USA) (ICER ’15). Association for Computing Machinery, New York, NY, USA, 141–150. https://doi.org/10.1145/2787622.2787710
[7]
Kevin Casey and David Azcona. 2017. Utilizing student activity patterns to predict performance. International Journal of Educational Technology in Higher Education 14, 1(2017), 1–15.
[8]
Suzanne L. Dazo, Nicholas R. Stepanek, Robert Fulkerson, and Brian Dorn. 2016. An Empirical Analysis of Video Viewing Behaviors in Flipped CS1 Courses. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education(Arequipa, Peru) (ITiCSE ’16). Association for Computing Machinery, New York, NY, USA, 106–111. https://doi.org/10.1145/2899415.2899468
[9]
Louis Deslauriers, Logan S. McCarty, Kelly Miller, Kristina Callaghan, and Greg Kestin. 2019. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences 116, 39(2019), 19251–19257. https://doi.org/10.1073/pnas.1821936116 arXiv:https://www.pnas.org/content/116/39/19251.full.pdf
[10]
Stephen H. Edwards, Jason Snyder, Manuel A. Pérez-Quiñones, Anthony Allevato, Dongkwan Kim, and Betsy Tretola. 2009. Comparing Effective and Ineffective Behaviors of Student Programmers. In Proceedings of the Fifth International Workshop on Computing Education Research Workshop (Berkeley, CA, USA) (ICER ’09). Association for Computing Machinery, New York, NY, USA, 3–14. https://doi.org/10.1145/1584322.1584325
[11]
Scott Freeman, Sarah L. Eddy, Miles McDonough, Michelle K. Smith, Nnadozie Okoroafor, Hannah Jordt, and Mary Pat Wenderoth. 2014. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences 111, 23(2014), 8410–8415. https://doi.org/10.1073/pnas.1319030111 arXiv:https://www.pnas.org/content/111/23/8410.full.pdf
[12]
Michail N. Giannakos, John Krogstie, and Nikos Chrisochoides. 2014. Reviewing the Flipped Classroom Research: Reflections for Computer Science Education. In Proceedings of the Computer Science Education Research Conference (Berlin, Germany) (CSERC ’14). Association for Computing Machinery, New York, NY, USA, 23–29. https://doi.org/10.1145/2691352.2691354
[13]
David Gross, Evava Pietri, Gordon Anderson, Karin Moyano Camihort, and Mark Graham. 2015. Increased Preclass Preparation Underlies Student Outcome Improvement in the Flipped Classroom. CBE life sciences education 14 (12 2015). https://doi.org/10.1187/cbe.15-02-0040
[14]
Leslie Harvey III and Ashish Aggarwal. 2021. Exploring the Effect of Quiz and Homework Submission Times on Students’ Performance in an Introductory Programming Course in a Flipped Classroom Environment. In 2021 ASEE Virtual Annual Conference Content Access. ASEE Conferences, Virtual Conference. https://peer.asee.org/37149
[15]
Arto Hellas, Petri Ihantola, Andrew Petersen, Vangel V. Ajanovski, Mirela Gutica, Timo Hynninen, Antti Knutas, Juho Leinonen, Chris Messom, and Soohyun Nam Liao. 2018. Predicting Academic Performance: A Systematic Literature Review(ITiCSE 2018 Companion). Association for Computing Machinery, New York, NY, USA, 175–199. https://doi.org/10.1145/3293881.3295783
[16]
Antti Herala, Erno Vanhala, Antti Knutas, and Jouni Ikonen. 2015. Teaching Programming with Flipped Classroom Method: A Study from Two Programming Courses. In Proceedings of the 15th Koli Calling Conference on Computing Education Research (Koli, Finland) (Koli Calling ’15). Association for Computing Machinery, New York, NY, USA, 165–166. https://doi.org/10.1145/2828959.2828983
[17]
Diane Horton, Michelle Craig, Jennifer Campbell, Paul Gries, and Daniel Zingaro. 2014. Comparing Outcomes in Inverted and Traditional CS1. In Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education (Uppsala, Sweden) (ITiCSE ’14). Association for Computing Machinery, New York, NY, USA, 261–266. https://doi.org/10.1145/2591708.2591752
[18]
Petri Ihantola, Arto Vihavainen, Alireza Ahadi, Matthew Butler, Jürgen Börstler, Stephen H. Edwards, Essi Isohanni, Ari Korhonen, Andrew Petersen, Kelly Rivers, Miguel Ángel Rubio, Judy Sheard, Bronius Skupas, Jaime Spacco, Claudia Szabo, and Daniel Toll. 2015. Educational Data Mining and Learning Analytics in Programming: Literature Review and Case Studies. In Proceedings of the 2015 ITiCSE on Working Group Reports (Vilnius, Lithuania) (ITICSE-WGR ’15). Association for Computing Machinery, New York, NY, USA, 41–63. https://doi.org/10.1145/2858796.2858798
[19]
Aliye Karabulut-Ilgu, Nadia Jaramillo Cherrez, and Charles T. Jahren. 2018. A systematic review of research on the flipped learning method in engineering education. British Journal of Educational Technology 49, 3 (2018), 398–411. https://doi.org/10.1111/bjet.12548
[20]
Hassan Khosravi and Kendra M.L. Cooper. 2017. Using Learning Analytics to Investigate Patterns of Performance and Engagement in Large Classes. In Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education (Seattle, Washington, USA) (SIGCSE ’17). Association for Computing Machinery, New York, NY, USA, 309–314. https://doi.org/10.1145/3017680.3017711
[21]
Päivi Kinnunen and Lauri Malmi. 2006. Why Students Drop out CS1 Course?. In Proceedings of the Second International Workshop on Computing Education Research (Canterbury, United Kingdom) (ICER ’06). Association for Computing Machinery, New York, NY, USA, 97–108. https://doi.org/10.1145/1151588.1151604
[22]
Celine Latulipe, Audrey Rorrer, and Bruce Long. 2018. Longitudinal Data on Flipped Class Effects on Performance in CS1 and Retention after CS1. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 411–416. https://doi.org/10.1145/3159450.3159518
[23]
Soohyun Nam Liao, Kartik Shah, William G. Griswold, and Leo Porter. 2021. A Quantitative Analysis of Study Habits Among Lower- and Higher-Performing Students in CS1. In Proceedings of the 26th ACM Conference on Innovation and Technology in Computer Science Education V. 1 (Virtual Event, Germany) (ITiCSE ’21). Association for Computing Machinery, New York, NY, USA, 366–372. https://doi.org/10.1145/3430665.3456350
[24]
Soohyun Nam Liao, Sander Valstar, Kevin Thai, Christine Alvarado, Daniel Zingaro, William G. Griswold, and Leo Porter. 2019. Behaviors of Higher and Lower Performing Students in CS1. In Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education (Aberdeen, Scotland Uk) (ITiCSE ’19). Association for Computing Machinery, New York, NY, USA, 196–202. https://doi.org/10.1145/3304221.3319740
[25]
Madeleine Lorås, Guttorm Sindre, Hallvard Trætteberg, and Trond Aalberg. 2021. Study Behavior in Computing Education—A Systematic Literature Review. ACM Trans. Comput. Educ. 22, 1, Article 9 (oct 2021), 40 pages. https://doi.org/10.1145/3469129
[26]
Charles Mcdowell, Linda Werner, Heather Bullock, and Julian Fernald. 2006. Pair programming improves student retention, confidence, and program quality. Commun. ACM 49 (08 2006), 90–95. https://doi.org/10.1145/1145293
[27]
Keir Mierle, Kevin Laven, Sam Roweis, and Greg Wilson. 2005. Mining Student CVS Repositories for Performance Indicators. SIGSOFT Softw. Eng. Notes 30, 4 (May 2005), 1–5. https://doi.org/10.1145/1082983.1083150
[28]
Colin Moore, Lina Battestilli, and Ignacio X. Domínguez. 2021. Finding Video-Watching Behavior Patterns in a Flipped CS1 Course. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (Virtual Event, USA) (SIGCSE ’21). Association for Computing Machinery, New York, NY, USA, 768–774. https://doi.org/10.1145/3408877.3432359
[29]
Cindy Norris. 2016. An Examination of Layers of Quizzing in Two Computer Systems Courses. In Proceedings of the 47th ACM Technical Symposium on Computing Science Education (Memphis, Tennessee, USA) (SIGCSE ’16). Association for Computing Machinery, New York, NY, USA, 48–53. https://doi.org/10.1145/2839509.2844608
[30]
Ted O’Donoghue and Matthew Rabin. 1999. Doing it now or later. American economic review 89, 1 (1999), 103–124.
[31]
Keith Quille and Susan Bergin. 2019. CS1: how will they do? How can we help? A decade of research and practice. Computer Science Education 29, 2-3 (2019), 254–282.
[32]
Sunil Sabnis, Renzhe Yu, and René F. Kizilcec. 2022. Large-Scale Student Data Reveal Sociodemographic Gaps in Procrastination Behavior. In Proceedings of the Ninth ACM Conference on Learning @ Scale (New York City, NY, USA) (L@S ’22). Association for Computing Machinery, New York, NY, USA, 133–141. https://doi.org/10.1145/3491140.3528285
[33]
Merilin Säde, Reelika Suviste, Piret Luik, Eno Tõnisson, and Marina Lepp. 2019. Factors That Influence Students’ Motivation and Perception of Studying Computer Science. In Proceedings of the 50th ACM Technical Symposium on Computer Science Education (Minneapolis, MN, USA) (SIGCSE ’19). Association for Computing Machinery, New York, NY, USA, 873–878. https://doi.org/10.1145/3287324.3287395
[34]
Clifford A. Shaffer and Stephen H. Edwards. 2011. Scheduling and Student Performance. In Proceedings of the 16th Annual Joint Conference on Innovation and Technology in Computer Science Education (Darmstadt, Germany) (ITiCSE ’11). Association for Computing Machinery, New York, NY, USA, 331. https://doi.org/10.1145/1999747.1999842
[35]
Jaime Spacco, Paul Denny, Brad Richards, David Babcock, David Hovemeyer, James Moscola, and Robert Duvall. 2015. Analyzing Student Work Patterns Using Programming Exercise Data. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education (Kansas City, Missouri, USA) (SIGCSE ’15). Association for Computing Machinery, New York, NY, USA, 18–23. https://doi.org/10.1145/2676723.2677297
[36]
Yaacov Trope and Ayelet Fishbach. 2000. Counteractive self-control in overcoming temptation.Journal of personality and social psychology 79, 4(2000), 493.
[37]
Christopher Watson and Frederick W.B. Li. 2014. Failure Rates in Introductory Programming Revisited. In Proceedings of the 2014 Conference on Innovation & Technology in Computer Science Education (Uppsala, Sweden) (ITiCSE ’14). Association for Computing Machinery, New York, NY, USA, 39–44. https://doi.org/10.1145/2591708.2591749
[38]
Chris Wilcox and Albert Lionelle. 2018. Quantifying the Benefits of Prior Programming Experience in an Introductory Computer Science Course. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education (Baltimore, Maryland, USA) (SIGCSE ’18). Association for Computing Machinery, New York, NY, USA, 80–85. https://doi.org/10.1145/3159450.3159480
[39]
Salla Willman, Rolf Lindén, Erkki Kaila, Teemu Rajala, Mikko-Jussi Laakso, and Tapio Salakoski. 2015. On study habits on an introductory course on programming. Computer Science Education 25, 3 (2015), 276–291. https://doi.org/10.1080/08993408.2015.1073829
[40]
Brenda Cantwell Wilson and Sharon Shrock. 2001. Contributing to Success in an Introductory Computer Science Course: A Study of Twelve Factors. SIGCSE Bull. 33, 1 (Feb. 2001), 184–188. https://doi.org/10.1145/366413.364581

Cited By

View all
  • (2024)A Comparison of Student Behavioral Engagement in Traditional Live Coding and Active Live Coding LecturesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653537(513-519)Online publication date: 3-Jul-2024
  • (2024)Academic procrastination, incentivized and self-selected spaced practice, and quiz performance in an online programming problem systemComputers & Education10.1016/j.compedu.2024.105029214:COnline publication date: 2-Jul-2024

Index Terms

  1. Exploring the Differences in Students’ Behavioral Engagement With Quizzes and Its Impact on their Performance in a Flipped CS1 Course

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Other conferences
      Koli Calling '22: Proceedings of the 22nd Koli Calling International Conference on Computing Education Research
      November 2022
      282 pages
      ISBN:9781450396165
      DOI:10.1145/3564721
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 17 November 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. CS1
      2. active learning
      3. flipped classroom
      4. self-regulation
      5. student behavior
      6. submission times
      7. time management

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      Koli 2022

      Acceptance Rates

      Overall Acceptance Rate 80 of 182 submissions, 44%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)21
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 06 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)A Comparison of Student Behavioral Engagement in Traditional Live Coding and Active Live Coding LecturesProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653537(513-519)Online publication date: 3-Jul-2024
      • (2024)Academic procrastination, incentivized and self-selected spaced practice, and quiz performance in an online programming problem systemComputers & Education10.1016/j.compedu.2024.105029214:COnline publication date: 2-Jul-2024

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media