skip to main content
10.1145/3626252.3630779acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article
Open access

Analyzing Differences in Student Engagement Between a Single Narrative Game Intervention and Multiple Narrative Games Intervention in an Undergraduate Computer Organization and Architecture Course

Published: 07 March 2024 Publication History

Abstract

Existing research strongly supports that educational video games have beneficial impacts on student engagement, motivation, and learning outcomes. However, more still needs to be done to understand the best ways to apply well designed educational games in classroom contexts. This study examines the differences in student engagement between two educational game approaches applied as extra credit opportunities in an online asynchronous computer organization and architecture course at a mid-western R1 university. The study evaluated the relative efficacy of a singular game with one overarching narrative versus multiple games with different narratives where the two treatments both cover the same learning content.
The results found no significant differences in engagement between the two groups. The findings are supported by a robust sample of quantitative gameplay data from 336 students across four semesters and two years of course offerings analyzed through a continuation ratio model. The results are additionally supported by pre/post-survey data from a subset of the same student participants that provided their gameplay data. This implies that sets of heterogeneous games can be as effective as a single homogeneous game that integrates all content. While the results found no significant differences in engagement, a pre-post test included in the study had results that suggested future research that should be done to more robustly examine differences in learning outcomes in the two application contexts.

References

[1]
Alan Agresti. 2003. Categorical data analysis. John Wiley & Sons.
[2]
Christopher Ball, Kuo-Ting Huang, Shelia R Cotten, and RV Rikard. 2020. Gaming the SySTEM: The relationship between video games and the digital and STEM divides. Games and Culture, Vol. 15, 5 (2020), 501--528.
[3]
Brianno D Coller and Michael J Scott. 2009. Effectiveness of using a video game to teach a course in mechanical engineering. Computers & Education, Vol. 53, 3 (2009), 900--912.
[4]
André Czauderna and Emmanuel Guardiola. 2019. The gameplay loop methodology as a tool for educational game design. Electronic Journal of e-Learning, Vol. 17, 3 (2019), pp207--221.
[5]
Sara De Freitas. 2018. Are games effective learning tools? A review of educational games. Journal of Educational Technology & Society, Vol. 21, 2 (2018), 74--84.
[6]
Joost CF De Winter and Dimitra Dodou. 2010. Five-point Likert items: t test versus Mann-Whitney-Wilcoxon. Practical assessment, research & evaluation, Vol. 15, 11 (2010), 1--12.
[7]
Michele D Dickey. 2011. Murder on Grimm Isle: The impact of game narrative design in an educational game-based learning environment. British journal of educational technology, Vol. 42, 3 (2011), 456--469.
[8]
Robin Hunicke, Marc LeBlanc, and Robert Zubek. 2004. MDA: A formal approach to game design and game research. In Proceedings of the AAAI Workshop on Challenges in Game AI, Vol. 4. San Jose, CA, 1722.
[9]
Wijnand A IJsselsteijn, Yvonne AW De Kort, and Karolien Poels. 2013. The game experience questionnaire. (2013).
[10]
Chaima Jemmali, Sara Bunian, Andrea Mambretti, and Magy Seif El-Nasr. 2018. Educational game design: an empirical study of the effects of narrative. In Proceedings of the 13th international conference on the foundations of digital games. 1--10.
[11]
Daniel Johnson, M John Gardner, and Ryan Perry. 2018. Validation of two game experience scales: the player experience of need satisfaction (PENS) and game experience questionnaire (GEQ). International Journal of Human-Computer Studies, Vol. 118 (2018), 38--46.
[12]
Marjaana Kangas, Antti Koskinen, and Leena Krokfors. 2017. A qualitative literature review of educational games in the classroom: the teacher's pedagogical activities. Teachers and Teaching, Vol. 23, 4 (2017), 451--470.
[13]
Fengfeng Ke. 2009. A qualitative meta-analysis of computer games as learning tools. Handbook of research on effective electronic gaming in education (2009), 1--32.
[14]
Nedjma Koval-Saifi and Jan Plass. 2018. Antura and the Letters: Impact and technical evaluation. (2018).
[15]
David R Krathwohl. 2002. A revision of Bloom's taxonomy: An overview. Theory into practice, Vol. 41, 4 (2002), 212--218.
[16]
Akiva M Liberman. 2005. How much more likely? The implications of odds ratios for probabilities. American Journal of Evaluation, Vol. 26, 2 (2005), 253--266.
[17]
Declan McClintock. 2022. All Together Now, Using Multiple Frameworks to Inform Serious Game Design and Development. In International Conference on Meaningful Play. https://meaningfulplay.msu.edu/proceedings2022/mp2022_paper_5725.pdf
[18]
Declan McClintock and Charles Owen. 2021. Common Narrative in Educational Video Games: A Design of Games to Teach Circuits. In The 16th International Conference on the Foundations of Digital Games (FDG) 2021. 1--4.
[19]
Declan McClintock and Charles B Owen. 2023. CircuBot Source Project. https://drive.google.com/drive/folders/1exmTMwWE6jMgPyXPvIHO06Q-ZlVmLU-x?usp=sharing
[20]
Michael A Miljanovic and Jeremy S Bradbury. 2018. A review of serious games for programming. In Joint international conference on serious games. Springer, 204--216.
[21]
Konstantin Mitgutsch and Narda Alvarado. 2012. Purposeful by design? A serious game design assessment framework. In Proceedings of the International Conference on the foundations of digital games. 121--128.
[22]
Emily Naul and Min Liu. 2020. Why story matters: A review of narrative in serious games. Journal of Educational Computing Research, Vol. 58, 3 (2020), 687--707.
[23]
Age of Learning. 2022. Age of Learning Products. https://www.ageoflearning.com/products/
[24]
R-core. 1969. stats package - RDocumentation. https://www.rdocumentation.org/packages/stats/versions/3.6.2
[25]
Josephine M Randel, Barbara A Morris, C Douglas Wetzel, and Betty V Whitehill. 1992. The effectiveness of games for educational purposes: A review of recent research. Simulation & gaming, Vol. 23, 3 (1992), 261--276.
[26]
Yves Rosseel. 2012. lavaan: An R package for structural equation modeling. Journal of statistical software, Vol. 48 (2012), 1--36.
[27]
Jonathan P Rowe, Lucy R Shores, Bradford W Mott, and James C Lester. 2011. Integrating learning, problem solving, and engagement in narrative-centered learning environments. International Journal of Artificial Intelligence in Education, Vol. 21, 1--2 (2011), 115--133.
[28]
Samuel Sanford Shapiro and Martin B Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika, Vol. 52, 3/4 (1965), 591--611.
[29]
Brian M Winn. 2009. The design, play, and experience framework. In Handbook of research on effective electronic gaming in education. IGI Global, 1010--1024.
[30]
Zhiyong Zhang, Yujiao Mai, and Miao Yang. 2021. WebPower: Basic and Advanced Statistical Power Analysis. https://cran.r-project.org/web/packages/WebPower/index.html
[31]
Zhiyong Zhang and Ke-Hai Yuan. 2018. Practical statistical power analysis using Webpower and R. Isdsa Press. io

Index Terms

  1. Analyzing Differences in Student Engagement Between a Single Narrative Game Intervention and Multiple Narrative Games Intervention in an Undergraduate Computer Organization and Architecture Course

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGCSE 2024: Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1
    March 2024
    1583 pages
    ISBN:9798400704239
    DOI:10.1145/3626252
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 March 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. computing education research
    2. digital game based learning
    3. educational video games
    4. serious games

    Qualifiers

    • Research-article

    Conference

    SIGCSE 2024
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,595 of 4,542 submissions, 35%

    Upcoming Conference

    SIGCSE TS 2025
    The 56th ACM Technical Symposium on Computer Science Education
    February 26 - March 1, 2025
    Pittsburgh , PA , USA

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 200
      Total Downloads
    • Downloads (Last 12 months)200
    • Downloads (Last 6 weeks)24
    Reflects downloads up to 24 Jan 2025

    Other Metrics

    Citations

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media