research-article

Modeling Metacomprehension Monitoring Accuracy with Eye Gaze on Informational Content in a Multimedia Learning Environment

Authors:

Megan D Wiedbusch,

Roger AzevedoAuthors Info & Claims

ETRA '20 Full Papers: ACM Symposium on Eye Tracking Research and Applications

Article No.: 20, Pages 1 - 9

https://doi.org/10.1145/3379155.3391329

Published: 02 June 2020 Publication History

Abstract

Multimedia learning environments support learners in developing self-regulated learning (SRL) strategies. However, capturing these strategies and cognitive processes can be difficult for researchers because cognition is often inferred, not directly measured. This study sought to model self-reported metacognitive judgments using eye-tracking from 60 undergraduate students as they learned about biological systems with MetaTutorIVH, a multimedia learning environment. We found that participants’ gaze behaviors were different between the perceived relevance of the instructional content provided regardless of the actual content relevance. Additionally, we fit a cumulative link mixed effects ordinal regression model to explain reported metacognitive judgments based on content fixations, relevance, and presentation type. Main effects were found for all variables and several interactions between both fixations and content relevance as well as content fixations and presentation type. Surprisingly, accurate metacognitive judgments did not explain performance. Implication for multimedia learning environment design are discussed.

References

[1]

Alan Agresti. 2018. An introduction to categorical data analysis. John Wiley & Sons, Hoboken, NJ.

[2]

Shaaron Ainsworth and Andrea Th Loizou. 2003. The effects of self-explaining when learning with text or diagrams. Cognitive science 27, 4 (2003), 669–681.

[3]

Ecenaz Alemdag and Kursat Cagiltay. 2018. A systematic review of eye tracking research on multimedia learning. Computers & Education 125 (2018), 413–428.

Digital Library

[4]

Alessandro Antonietti, Barbara Colombo, and Chiara Di Nuzzo. 2015. Metacognition in self-regulated multimedia learning: Integrating behavioural, psychophysiological and introspective measures. Learning, Media and Technology 40, 2 (2015), 187–209.

[5]

N Ariasi, Jukka Hyönä, Johanna K Kaakinen, and Lucia Mason. 2017. An eye-movement analysis of the refutation effect in reading science text. Journal of Computer Assisted Learning 33, 3 (2017), 202–221.

[6]

Robert Ariel, John Dunlosky, and Heather Bailey. 2009. Agenda-based regulation of study-time allocation: when agendas override item-based monitoring.Journal of Experimental Psychology: General 138, 3 (2009), 432.

[7]

Roger Azevedo. 2014. Issues in dealing with sequential and temporal characteristics of self-and socially-regulated learning. Metacognition and Learning 9, 2 (2014), 217–228.

[8]

Roger Azevedo and Jennifer G Cromley. 2004. Does training on self-regulated learning facilitate students’ learning with hypermedia?Journal of educational psychology 96, 3 (2004), 523.

[9]

Roger Azevedo, Jeffrey A Greene, and Daniel C Moos. 2007. The effect of a human agent’s external regulation upon college students’ hypermedia learning. Metacognition and learning 2, 2-3 (2007), 67–87.

[10]

Roger Azevedo, Nicholas V Mudrick, Michelle Taub, and Amanda Bradbury. 2019. Self-Regulation in Computer-Assisted Learning Systems. In The Cambridge Handbook of Cognition and Education, John Dunlosky and Katherine A Rawson (Eds.). Cambridge University Press, Cambridge,United Kingdom, Chapter 23, 587–715.

[11]

Maria Bannert, Melanie Hildebrand, and Christoph Mengelkamp. 2009. Effects of a metacognitive support device in learning environments. Computers in human behavior 25, 4 (2009), 829–835.

[12]

Monique Boekaerts. 2011. Emotions, emotion regulation, and self-regulation of learning. In Handbook of self-regulation of learning and performance, Dale H Schunk and Barry Zimmerman (Eds.). Vol. 5. Routledge, New York, NY, 408–425.

[13]

Victoria Brown, Bennett L Schwartz, and Anastasia Efklides. 2017. Motivation and Affect in Self-Regulated Learning: Does Metacognition Play a Role?In Handbook of Self-Regulation of Learning and Performance. Routledge, New York, NY, 80–98.

[14]

Candice Burkett and Roger Azevedo. 2012. The effect of multimedia discrepancies on metacognitive judgments. Computers in Human Behavior 28, 4 (2012), 1276–1285.

Digital Library

[15]

Kirsten R Butcher. 2014. The multimedia principle. In The Cambridge handbook of multimedia learning. Vol. 2. Cambridge University Press New York, New York, NY, 174–205.

[16]

Leen Catrysse, David Gijbels, Vincent Donche, Sven De Maeyer, Marije Lesterhuis, and Piet Van den Bossche. 2018. How are learning strategies reflected in the eyes? Combining results from self-reports and eye-tracking. British journal of educational psychology 88, 1 (2018), 118–137.

[17]

Philip I Chow and Howard Berenbaum. 2012. Perceived utility of emotion: The structure and construct validity of the Perceived Affect Utility Scale in a cross-ethnic sample.Cultural Diversity and Ethnic Minority Psychology 18, 1(2012), 55.

[18]

R. H. B. Christensen. 2019. ordinal—Regression Models for Ordinal Data. R package version 2019.12-10. https://CRAN.R-project.org/package=ordinal.

[19]

Hester de Boer, Anouk S. Kostons, and Donker-Bergstra Danny D. N. M.2012. Effective Strategies for Self-regulated Learning : A Meta-Analysis. Ph.D. Dissertation. Gronings Instituut voor Onderzoek van Onderwijs,.

[20]

Charlotte Dignath and Gerhard Büttner. 2008. Components of fostering self-regulated learning among students . A meta-analysis on intervention studies at primary and secondary school level. Metacognition Learning 3, 3 (2008), 231–264.

[21]

John Dunlosky and Janet Metcalfe. 2008. Metacognition. Sage Publications, Thousand Oaks, CA.

[22]

John Dunlosky and Katherine A Rawson. 2019. The Cambridge Handbook of Cognition and Education. Cambridge University Press, New York, NY.

[23]

John Dunlosky, Katherine A Rawson, and Douglas J Hacker. 2002. 11. Metacomprehension of science text: investigating the levels of disruption hypothesis. In The psychology of science text comprehension. Routledge, New York, NY, 255–280.

[24]

John Dunlosky, Katherine A Rawson, and Erica L Middleton. 2005. What constrains the accuracy of metacomprehension judgments ? Testing the transfer-appropriate-monitoring and accessibility hypotheses. Jounral of Memory and Language 52 (2005), 551–565.

[25]

Jeffrey Alan Greene and Roger Azevedo. 2007. A theoretical review of Winne and Hadwin ’ s model of self-regulated learning : New perspectives and directions. Review of Educational Research 77, 3 (2007), 334–372.

[26]

Jeffrey Alan Greene and Roger Azevedo. 2009. A macro-level analysis of SRL processes and their relations to the acquisition of a sophisticated mental model of a complex system. Contemporary Educational Psychology 34 (2009), 18–29.

[27]

Thomas D Griffin, Jennifer Wiley, and Keith W Thiede. 2008. Individual differences, rereading, and self-explanation : Concurrent processing and cue validity as constraints on metacomprehension accuracy. Memory & Cognition 36, 1 (2008), 93–103.

[28]

James J Gross and Oliver P John. 2003. Individual Differences in two emotion regulation processes : Implications for affect, relationships, and well-being. Journal of Personality and Social Psychology 85, 2(2003), 348–362.

[29]

Jukka Hyona and Robert F Lorch. 2004. Effects of topic headings on text processing : evidence from adult readers ’ eye fixation patterns. Learning and Instruction 14 (2004), 131–152.

[30]

iMotions. 2016. Attention Tool (version 6.2). apparatus and software.

[31]

Allison J Jaeger and Jennifer Wiley. 2014. Do illustrations help or harm metacomprehension accuracy ?Learning and Instruction 34 (2014), 58–73.

[32]

Renée S Jansen, Anouschka Van Leeuwen, Jeroen Janssen, and Suzanne Jak. 2019. Self-regulated learning partially mediates the effect of self- regulated learning interventions on achievement in higher education : A meta-analysis. Educational Research Review 28, September (2019), 1–20.

[33]

Johanna K Kaakinen and Jukka Hyo. 2010. Task effects on eye movements during reading. Journal of Experimental Psychology: Learning, Memory, & Cogntion 36, 6(2010), 1561–1566.

[34]

Asher Koriat. 1997. Monitoring one’s own knowledge during study: A cue-utilization approach to judgments of learning.Journal of experimental psychology: General 126, 4 (1997), 349.

[35]

Chwee Beng Lee. 2010. Generating synergy between conceptual change and knowledge building. Human Development 637616(2010), 134–152.

[36]

Hsueh-hua Chuang Han-chin Liu. 2012. Effects of different multimedia presentations on viewers ’ information-processing activities measured by eye-tracking technology. Journal of Science Education and Technology 2, 21 (2012), 276–286.

[37]

Richard E Mayer. 2014. Principles based on social cues in multimedia learning: Personalization, voice, image, and embodiment principles. The Cambridge handbook of multimedia learning 16 (2014), 345–370.

[38]

Mirjamaija Mikkilä-Erdmann, Marjaana Penttinen, Erkki Anto, and Erkki Olkinuora. 2008. Constructing mental models during learning from science text. In Understanding models for learning and instruction, Dirk Ifenthaler, Pablo J Pirnay-Dummer, and Michael Spector (Eds.). Springer, Springer, Boston, MA, 63–79.

[39]

Nicholas V Mudrick, Roger Azevedo, and Michelle Taub. 2019. Integrating metacognitive judgments and eye movements using sequential pattern mining to understand processes underlying multimedia learning. Computers in Human Behavior 96 (2019), 1–12.

Digital Library

[40]

Reinhard Pekrun, Thomas Goetz, Anne C Frenzel, Petra Barchfeld, and Raymond P Perry. 2011. Measuring emotions in students ’ learning and performance : The Achievement Emotions Questionnaire (AEQ). Contemporary Educational Psychology 36, 1 (2011), 36–48.

[41]

Marjaana Penttinen, Erkki Anto, and Mirjamaija Mikkila-Erdmann. 2013. Conceptual change, text comprehension and eye movements during reading. Research in Science Education 43 (2013), 1407–1434.

[42]

Nancy E Perry. 2010. Introduction : Using Qualitative Methods to Enrich Understandings of Self-Regulated Learning Introduction : Using qualitative methods to enrich understandings of self-regulated learning. Educational Psychologist 37 (2010), 1–3.

[43]

Nancy E Perry and Philip H Winne. 2006. Learning from learning kits : gStudy traces of students ’ self-regulated engagements with computerized content. Educational Psychologist Review 18 (2006), 211–228.

[44]

Celeste Pilegard and Richard E Mayer. 2015. Adding judgments of understanding to the metacognitive toolbox ☆. Learning and Individual Differences 41 (2015), 62–72.

[45]

Paul R Pintrich, Anastasia Danos Elder, Teresa Garcia, Lynley Hicks Anderman, Barbara Hofer, Helen Patrick, Allison Ryan, and Tim Urdan. 2000. Multiple goals, multiple pathways : The role of goal orientation in learning and achievement. Journal of Educational Psychology 92, 3 (2000), 544–555.

[46]

Keith Rayner. 2009. The 35th Sir Frederick Bartlett lecture eye movements and attention in reading, scene perception, and visual search. The Quarterly Journal of Experimental Psychology 62, 8(2009), 1457–1506.

[47]

Pedro Rodrigues and Pedro J Rosa. 2017. Eye-tracking as a research methodology in educational context: a spanning framework. In Eye-tracking technology applications in educational research, Christopher Was, Frank Sansosti, and Bradley Morris(Eds.). IGI Global, Hershey, PA, 1–26.

[48]

Dale H Schunk and Jeffrey A Greene. 2018. Handbook of Self-Regulation of Learning and Performance. Routledge, New York, NY.

[49]

Judith Schweppe and Ralf Rummer. 2016. Computers in human behavior integrating written text and graphics as a desirable difficulty in long-term multimedia learning. Computers in Human Behavior 60 (2016), 131–137.

Digital Library

[50]

Michael J Serra and John Dunlosky. 2005. Does retrieval fluency contribute to the underconfidence-with-practice effect?Journal of Experimental Psychology: Learning, Memory, & Cogntion 31, 6(2005), 1258–1266.

[51]

Keith W Thiede and Anique BH de Bruin. 2017. Self-regulated learning in reading. In Handbook of self-regulation of learning and performance, Dale H Schunk and Jeffrey A Greene (Eds.). Routledge, New York, NY, 140–153.

[52]

Tamara van Gog and Halszka Jarodzka. 2013. Eye tracking as a tool to study and enhance cognitive and metacognitive processes in computer-based learning environments. In International handbook of metacognition and learning technologies, Roger Azevedo and Vincent Aleven (Eds.). Springer, New York, NY, 143–156.

[53]

Marcel V J Veenman, Laura Bavelaar, Levina De Wolf, and Marieke G P Van Haaren. 2014. The on-line assessment of metacognitive skills in a computerized learning environment. Learning and Individual Differences 29 (2014), 123–130.

[54]

Nathan Weber and Neil Brewer. 2003. The effect of judgment type and confidence scale on confidence-accuracy calibration in face recognition.Journal of Applied Psychology 88, 3 (2003), 490.

[55]

Philip H Winne. 2017. Learning analytics for self-regulated learning. In Handbook of learning analytics, Dale H Schunk and Jeffrey A Greene (Eds.). The Society for Learning Analytics Research, Beaumont, AB, 241–249.

[56]

Philip H Winne, Dianne Jamieson-Noel, and Krista Muis. 2002. Methodological issues and advances in researching tactics, strategies, and self-regulated learning. Advances in motivation and achievement: New directions in measures and methods 12(2002), 121–155.

[57]

Menahem Yeari, Marja Oudega, and Paul van den Broek. 2017. The effect of highlighting on processing and memory of central and peripheral text information : evidence from eye movements. Journal of Research in Reading 40, 4 (2017), 365–383.

[58]

Barry J Zimmerman and Adam R Moylan. 2009. Self-regulation: Where metacognition and motivation intersect. In Handbook of metacognition in education, Douglas J Hacker, John Dunlosky, and Arthur C Graesser(Eds.). Routledge, New York, NY, 311–328.

Cited By

Whitehead RNguyen AJärvelä S(2024)Exploring the role of gaze behaviour in socially shared regulation of collaborative learning in a group taskJournal of Computer Assisted Learning10.1111/jcal.13022Online publication date: 2-Jun-2024
https://doi.org/10.1111/jcal.13022
Aksela OLämsä JJärvelä S(2024)Secondary School Students’ Enacted Self-Regulated Learning Strategies in a Computer-Based Writing Task–Insights from Digital Trace Data and InterviewsTechnology, Knowledge and Learning10.1007/s10758-024-09789-4Online publication date: 28-Oct-2024
https://doi.org/10.1007/s10758-024-09789-4
Brown MWiedbusch MPatel MNaderi ECapello SLlinas AAzevedo RMargondai A(2024)Designing for Self-Regulated Learning: A Dual-View Intelligent Visualization Dashboard to Support Instructors and Students Using Multimodal Trace Data in ClassroomsHCI International 2024 Posters10.1007/978-3-031-61953-3_2(9-19)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61953-3_2
Show More Cited By

Recommendations

Understanding metacomprehension accuracy within video annotation systems

While much research has been conducted on annotations and note-taking in the context of learning from static text and images, the influence of annotating instructional video upon metacognitive monitoring and learning remains unclear. This is an ...
Integrating metacognitive judgments and eye movements using sequential pattern mining to understand processes underlying multimedia learning
Abstract
Metacomprehension is key to successful learning of complex topics when using multimedia materials. The goal of this study was to determine if eye-movement dyads could be: (1) identified by sequence mining techniques, and (2) aligned ...
Highlights
- Aligned metacomprehension, eye movements, and multimedia learning processes.
- ...
Boosting metacomprehension accuracy in computer-supported learning

Although learners' judgments of their own learning are crucial for self-regulated study, judgment accuracy tends to be low. To increase accuracy, we had participants make combined judgments. In Experiment 1, 247 participants studied a ten-chapter ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ETRA '20 Full Papers: ACM Symposium on Eye Tracking Research and Applications

June 2020

214 pages

ISBN:9781450371339

DOI:10.1145/3379155

Editors:
Andreas Bulling
University of Stuttgart, Germany
,
Anke Huckauf
Ulm University, Germany
,
Eakta Jain
University of Florida, USA
,
Ralph Radach
University of Wuppertal, Germany
,
Daniel Weiskopf
University of Stuttgart, Germany

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 June 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

National Science Foundation

Conference

ETRA '20

Sponsor:

ETRA '20: 2020 Symposium on Eye Tracking Research and Applications

June 2 - 5, 2020

Stuttgart, Germany

Acceptance Rates

Overall Acceptance Rate 69 of 137 submissions, 50%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
418
Total Downloads

Downloads (Last 12 months)38
Downloads (Last 6 weeks)2

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Whitehead RNguyen AJärvelä S(2024)Exploring the role of gaze behaviour in socially shared regulation of collaborative learning in a group taskJournal of Computer Assisted Learning10.1111/jcal.13022Online publication date: 2-Jun-2024
https://doi.org/10.1111/jcal.13022
Aksela OLämsä JJärvelä S(2024)Secondary School Students’ Enacted Self-Regulated Learning Strategies in a Computer-Based Writing Task–Insights from Digital Trace Data and InterviewsTechnology, Knowledge and Learning10.1007/s10758-024-09789-4Online publication date: 28-Oct-2024
https://doi.org/10.1007/s10758-024-09789-4
Brown MWiedbusch MPatel MNaderi ECapello SLlinas AAzevedo RMargondai A(2024)Designing for Self-Regulated Learning: A Dual-View Intelligent Visualization Dashboard to Support Instructors and Students Using Multimodal Trace Data in ClassroomsHCI International 2024 Posters10.1007/978-3-031-61953-3_2(9-19)Online publication date: 1-Jun-2024
https://doi.org/10.1007/978-3-031-61953-3_2
Dever DSonnenfeld NWiedbusch MSchmorrow SAmon MAzevedo R(2023)A complex systems approach to analyzing pedagogical agents’ scaffolding of self-regulated learning within an intelligent tutoring systemMetacognition and Learning10.1007/s11409-023-09346-x18:3(659-691)Online publication date: 19-May-2023
https://doi.org/10.1007/s11409-023-09346-x
Wiedbusch MLester JAzevedo R(2023)A multi-level growth modeling approach to measuring learner attention with metacognitive pedagogical agentsMetacognition and Learning10.1007/s11409-023-09336-z18:2(465-494)Online publication date: 3-Mar-2023
https://doi.org/10.1007/s11409-023-09336-z
Wiedbusch MDever DLi SAmon MLajoie SAzevedo R(2023)Measuring Multidimensional Facets of SRL Engagement with Multimodal DataUnobtrusive Observations of Learning in Digital Environments10.1007/978-3-031-30992-2_10(141-173)Online publication date: 14-Jun-2023
https://doi.org/10.1007/978-3-031-30992-2_10
Azevedo RBouchet FDuffy MHarley JTaub MTrevors GCloude EDever DWiedbusch MWortha FCerezo R(2022)Lessons Learned and Future Directions of MetaTutor: Leveraging Multichannel Data to Scaffold Self-Regulated Learning With an Intelligent Tutoring SystemFrontiers in Psychology10.3389/fpsyg.2022.81363213Online publication date: 14-Jun-2022
https://doi.org/10.3389/fpsyg.2022.813632
Wiedbusch MSonnenfeld NDever DAzevedo R(2022)Clustering Learner’s Metacognitive Judgment Accuracy and Bias to Explore Learning with AIEd SystemsArtificial Intelligence in Education10.1007/978-3-031-11644-5_33(402-413)Online publication date: 27-Jul-2022
https://doi.org/10.1007/978-3-031-11644-5_33
Cloude EWiedbusch MDever DTorre DAzevedo R(2022)The Role of Metacognition and Self-regulation on Clinical Reasoning: Leveraging Multimodal Learning Analytics to Transform Medical EducationThe Multimodal Learning Analytics Handbook10.1007/978-3-031-08076-0_5(105-129)Online publication date: 31-May-2022
https://doi.org/10.1007/978-3-031-08076-0_5
Koorathota SThakoor KHong LMao YAdelman PSajda P(2021)A Recurrent Neural Network for Attenuating Non-cognitive Components of Pupil DynamicsFrontiers in Psychology10.3389/fpsyg.2021.60452212Online publication date: 1-Feb-2021
https://doi.org/10.3389/fpsyg.2021.604522
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents