research-article

The role of achievement goal orientations when studying effect of learning analytics visualizations

Authors:

Sanam Shirazi Beheshitha,

Dragan Gašević,

Srećko JoksimovićAuthors Info & Claims

LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge

Pages 54 - 63

https://doi.org/10.1145/2883851.2883904

Published: 25 April 2016 Publication History

Abstract

When designing learning analytics tools for use by learners we have an opportunity to provide tools that consider a particular learner's situation and the learner herself. To afford actual impact on learning, such tools have to be informed by theories of education. Particularly, educational research shows that individual differences play a significant role in explaining students' learning process. However, limited empirical research in learning analytics has investigated the role of theoretical constructs, such as motivational factors, that are underlying the observed differences between individuals. In this work, we conducted a field experiment to examine the effect of three designed learning analytics visualizations on students' participation in online discussions in authentic course settings. Using hierarchical linear mixed models, our results revealed that effects of visualizations on the quantity and quality of messages posted by students with differences in achievement goal orientations could either be positive or negative. Our findings highlight the methodological importance of considering individual differences and pose important implications for future design and research of learning analytics visualizations.

References

[1]

Akyol, Z. and Garrison, D. R. 2011. Understanding cognitive presence in an online and blended community of inquiry: Assessing outcomes and processes for deep approaches to learning. British J. of Educational Technology. 42, 2, 233--250.

[2]

Arnold, K. and Pistilli, M. 2012. Course signals at Purdue: using learning analytics to increase student success. The 2nd Int. Conf. on Learning Analytics and Knowledge, 267--270.

Digital Library

[3]

Biggs, J. B. 1987. Student Approaches to Learning and Studying. Research Monograph. ERIC.

[4]

Brusilovsky, P., Hsiao, I.-H. and Folajimi, Y. 2011. QuizMap: open social student modeling and adaptive navigation support with TreeMaps. Towards Ubiq. Learning. 71--82.

Digital Library

[5]

Bull, S. and Kay, J. 2008. Metacognition and open learner models. The 3rd Workshop on Meta-Cognition and Self-Regulated Learning in Educational Technologies, at ITS2008, 7--20.

[6]

Corrin, L. and de Barba, P. 2014. Exploring students' interpretation of feedback delivered through learning analytics dashboards. Proc. of the ascilite 2014 conference, 201--205.

[7]

Dawson, S., Bakharia, A. and Heathcote, E. 2010. SNAPP: Realising the affordances of real-time SNA within networked learning environments. Proc. of the 7th Int. Conf. on Networked Learning, 125--133.

[8]

Dimitrova, V. 2003. STyLE-OLM: Interactive open learner modelling. Int. Journal of Artificial Intelligence in Education (IJAIED). 13, 35--78.

Digital Library

[9]

Dowell, N. M., Cade, W. L., Tausczik, Y., Pennebaker, J. and Graesser, A. C. 2014. What works: Creating adaptive and intelligent systems for collaborative learning support. Intelligent Tutoring Systems, 124--133.

[10]

Dowell, N. M., Graesser, A. C. and Cai, Z. Language and Discourse Analysis with Coh-Metrix: Applications from Educational Material to Learning Environments at Scale, Journal of Learning Analytics, in press.

[11]

Elliot, A. J. 1999. Approach and avoidance motivation and achievement goals. Educ. Psychologist. 34, 3, 169--189.

[12]

Elliot, A. J., Elliot, A. J. and Dweck, C. S. 2005. A conceptual history of the achievement goal construct. Handbook of Competence and Motivation. 16, 52--72.

[13]

Elliot, A. J., Murayama, K. and Pekrun, R. 2011. A 3×2 achievement goal model. Journal of Educational Psychology. 103, 3, 632.

[14]

Foltz, P. W., Kintsch, W. and Landauer, T. K. 1998. The measurement of textual coherence with latent semantic analysis. Discourse Processes. 25, 2-3, 285--307.

[15]

Friedman, J., Hastie, T. and Tibshirani, R. 2001. The elements of statistical learning. Springer series in statistics Springer, Berlin.

[16]

Gašević, D., Adesope, O., Joksimović, S. and Kovanović, V. 2015. Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions. The Internet and Higher Education. 24, 53--65.

[17]

Govaerts, S., Verbert, K., Duval, E. and Pardo, A. 2012. The student activity meter for awareness and self-reflection. CHI'12 Extended Abstracts on Human Factors in Computing Systems, 869--884.

Digital Library

[18]

Graesser, A. C., McNamara, D. S. and Kulikowich, J. M. 2011. Coh-Metrix providing multilevel analyses of text characteristics. Educational Researcher. 40, 5, 223--234.

[19]

Joksimovic, S., Gasevic, D., Kovanovic, V., Adesope, O. and Hatala, M. 2014. Psychological characteristics in cognitive presence of communities of inquiry: A linguistic analysis of online discussions. The Internet and Higher Education. 22, 1--10.

[20]

Kanuka, H. and Anderson, T. 2007. Online social interchange, discord, and knowledge construction. International Journal of E-Learning & Distance Education. 13, 1, 57--74.

[21]

Kay, J., Maisonneuve, N., Yacef, K. and Reimann, P. 2006. The big five and visualisations of team work activity. Proc. of the 8th Int. Conf. on Intel. Tutoring Systems, 197--206.

Digital Library

[22]

Kerly, A., Ellis, R. and Bull, S. 2008. CALMsystem: a conversational agent for learner modelling. Knowledge-Based Systems. 21, 3, 238--246.

Digital Library

[23]

Kruse, A. and Pongsajapan, R. 2012. Student-centered learning analytics. CNDLS Thought Papers, 1--9.

[24]

Leony, D., Pardo, A., de la Fuente Valentín, L., de Castro, D. S. and Kloos, C. D. 2012. GLASS: a learning analytics visualization tool. Proc. of the 2nd Int. Conf. on Learning Analytics and Knowledge, 162--163.

Digital Library

[25]

Luppicini, R. 2007. Review of computer mediated communication research for education. Instructional Science. 35, 2, 141--185.

[26]

Maehr, M. L. 1989. Thoughts about motivation. Research on motivation in education: Goals and cognitions. 3, 1, 299--315.

[27]

Mazza, R. and Milani, C. 2004. Gismo: a graphical interactive student monitoring tool for course management systems. Int. Conf. on Technology-Enhanced Learning, 1--8.

[28]

Nakahara, J., Hisamatsu, S., Yaegashi, K. and Yamauchi, Y. 2005. iTree: Does the mobile phone encourage learners to be more involved in collaborative learning? Proc. of the Conf. on Computer Support for Collaborative Learning: Learning 2005: the next 10 years!, 470--478.

Digital Library

[29]

Rovai, A. P. 2007. Facilitating online discussions effectively. The Internet and Higher Education. 10, 1, 77--88.

[30]

Santos, J. L., Verbert, K., Govaerts, S. and Duval, E. 2013. Addressing learner issues with StepUp!: an Evaluation. The 3rd Int. Conf. on Learning Analytics and Knowledge, 14--22.

Digital Library

[31]

Schielzeth, H. and Nakagawa, S. 2013. Nested by design: model fitting and interpretation in a mixed model era. Methods in Ecology and Evolution. 4, 1, 14--24.

[32]

Senko, C., Hulleman, C. S. and Harackiewicz, J. M. 2011. Achievement goal theory at the crossroads: Old controversies, current challenges, and new directions. Educational Psychologist. 46, 1, 26--47.

[33]

Shum, S. B. and Ferguson, R. 2012. Social learning analytics. Journal of Educational Technology & Society. 15, 3, 3--26.

[34]

Snow, R. E. 1991. Aptitude-treatment interaction as a framework for research on individual differences in psychotherapy. J. of Consulting and Clinical Psychology. 59, 2, 205--216.

[35]

Verbert, K., Duval, E., Klerkx, J., Govaerts, S. and Santos, J. L. 2013. Learning analytics dashboard applications. American Behavioral Scientist. 57, 10, 1500--1509.

[36]

Winne, P. H. 2010. Improving measurements of self-regulated learning. Educ. Psychologist. 45, 4, 267--276.

[37]

Winne, P. H. and Hadwin, A. F. 1998. Studying as self-regulated learning. Metacognition in Educational Theory and Practice. 93, 27--30.

[38]

Wise, A. F. 2014. Designing pedagogical interventions to support student use of learning analytics. Proc. of the 4th Int. Conf. on Learning Analytics and Knowledge, 203--211.

Digital Library

[39]

Wise, A. F., Hausknecht, S. N. and Zhao, Y. 2014. Attending to others' posts in asynchronous discussions: Learners' online "listening" and its relationship to speaking. Int. Journal of Computer-Supported Collaborative Learning. 9, 2, 185--209.

[40]

Wise, A. F., Speer, J., Marbouti, F. and Hsiao, Y.-T. 2013. Broadening the notion of participation in online discussions: examining patterns in learners' online listening behaviors. Instructional Science. 41, 2, 323--343.

[41]

Wise, A., Zhao, Y. and Hausknecht, S. 2014. Learning analytics for online discussions: Embedded and extracted approaches. Journal of Learning Analytics. 1, 2, 48--71.

[42]

Xu, R. 2003. Measuring explained variation in linear mixed effects models. Statistics in Medicine. 22, 22, 3527--3541.

[43]

Yuan, J. and Kim, C. 2014. Guidelines for facilitating the development of learning communities in online courses. Journal of Computer-Assisted Learning. 30, 3, 220--232.

Digital Library

Cited By

Wang ABorland DPeck TWang WGotz D(2024)Causal Priors and Their Influence on Judgements of Causality in Visualized DataIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345638131:1(765-775)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456381
Wu JLiao CTsai CKwok O(2024)Using learning analytics with temporal modeling to uncover the interplay of before-class video viewing engagement, motivation, and performance in an active learning contextComputers & Education10.1016/j.compedu.2023.104975212:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.compedu.2023.104975
Jovanović JZamecnik ABarthakur ADawson S(2024)Curriculum analytics: Exploring assessment objectives, types, and grades in a study programEducation and Information Technologies10.1007/s10639-024-13015-0Online publication date: 6-Sep-2024
https://doi.org/10.1007/s10639-024-13015-0
Show More Cited By

Index Terms

The role of achievement goal orientations when studying effect of learning analytics visualizations
1. Applied computing
  1. Education
    1. Distance learning
    2. E-learning

Recommendations

Towards the development of a learning analytics extension in open edX
TEEM '14: Proceedings of the Second International Conference on Technological Ecosystems for Enhancing Multiculturality

The emergence of platforms to support MOOCs (Massive Open Online Courses) strengthens the need of a powerful learning analytics support since teachers cannot be aware of so many students. However, the learning analytics support in MOOC platforms is in ...
Using Learning Analytics to Promote Student Engagement and Achievement in Blended Learning: An Empirical Study
ICEBT '18: Proceedings of the 2018 2nd International Conference on E-Education, E-Business and E-Technology

The emergence of blended learning has huge impact on traditional learning. Blended learning has its own unique characteristics combining the advantages of traditional learning and online learning. However, some problems of blended learning have also ...
An architecture for extending the learning analytics support in the Khan Academy framework
TEEM '13: Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality

The Khan Academy platform enables powerful on-line courses in which students can watch videos, solve exercises or earn badges. This platform provides an advanced learning analytics module with useful visualizations for teachers and students. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & Knowledge

April 2016

567 pages

ISBN:9781450341905

DOI:10.1145/2883851

General Chairs:
Dragan Gašević
University of Edinburgh, United Kingdom
,
Grace Lynch
Society for Learning Analytics Research
,
Program Chairs:
Shane Dawson
University of South Australia
,
Hendrik Drachsler
University of the Netherlands
,
Carolyn Penstein Rosé
Carnegie Mellon University

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

LAK '16

LAK '16: 6th International Conference on Learning Analytics and Knowledge

April 25 - 29, 2016

Edinburgh, United Kingdom

Acceptance Rates

LAK '16 Paper Acceptance Rate 36 of 116 submissions, 31%;

Overall Acceptance Rate 236 of 782 submissions, 30%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

40
Total Citations
View Citations
993
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)2

Reflects downloads up to 02 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang ABorland DPeck TWang WGotz D(2024)Causal Priors and Their Influence on Judgements of Causality in Visualized DataIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345638131:1(765-775)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456381
Wu JLiao CTsai CKwok O(2024)Using learning analytics with temporal modeling to uncover the interplay of before-class video viewing engagement, motivation, and performance in an active learning contextComputers & Education10.1016/j.compedu.2023.104975212:COnline publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1016/j.compedu.2023.104975
Jovanović JZamecnik ABarthakur ADawson S(2024)Curriculum analytics: Exploring assessment objectives, types, and grades in a study programEducation and Information Technologies10.1007/s10639-024-13015-0Online publication date: 6-Sep-2024
https://doi.org/10.1007/s10639-024-13015-0
Kleimola RHirsto LRuokamo H(2024)Promoting higher education students’ self-regulated learning through learning analytics: A qualitative studyEducation and Information Technologies10.1007/s10639-024-12978-4Online publication date: 7-Sep-2024
https://doi.org/10.1007/s10639-024-12978-4
Bahari MArpaci IAzmi NShuib L(2023)Predicting the Intention to Use Learning Analytics for Academic Advising in Higher EducationSustainability10.3390/su15211519015:21(15190)Online publication date: 24-Oct-2023
https://doi.org/10.3390/su152115190
Wang JAlKadi MBach B(2023)Show Me My Users: A Dashboard Visualizing User Interaction Logs2023 IEEE Visualization and Visual Analytics (VIS)10.1109/VIS54172.2023.00040(156-160)Online publication date: 21-Oct-2023
https://doi.org/10.1109/VIS54172.2023.00040
Liao CWu J(2023)Learning analytics on video-viewing engagement in a flipped statistics courseComputers & Education10.1016/j.compedu.2023.104754197:COnline publication date: 1-May-2023
https://dl.acm.org/doi/10.1016/j.compedu.2023.104754
Ahmad ASchneider JGriffiths DBiedermann DSchiffner DGreller WDrachsler H(2022)Connecting the dots – A literature review on learning analytics indicators from a learning design perspectiveJournal of Computer Assisted Learning10.1111/jcal.1271640:6(2432-2470)Online publication date: 26-Jul-2022
https://doi.org/10.1111/jcal.12716
Maag AWithana CBudhathoki SAlsadoon AVO T(2022)Learner-facing learning analytic – Feedback and motivation: A critiqueLearning and Motivation10.1016/j.lmot.2021.10176477(101764)Online publication date: Feb-2022
https://doi.org/10.1016/j.lmot.2021.101764
Akazaki JMachado LBehar P(2022)Socio-affective Profiles in Virtual Learning Environments: Using Learning AnalyticsInnovative Approaches to Technology-Enhanced Learning for the Workplace and Higher Education10.1007/978-3-031-21569-8_2(15-26)Online publication date: 16-Dec-2022
https://doi.org/10.1007/978-3-031-21569-8_2
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten