skip to main content
10.1145/3580305.3599407acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Learning Behavior-oriented Knowledge Tracing

Published: 04 August 2023 Publication History

Abstract

Exploring how learners' knowledge states evolve during the learning activities is a critical task in online learning systems, which can facilitate personalized services downstream, such as course recommendation. Most of existing methods have devoted great efforts to analyzing learners' knowledge states according to their responses (i.e., right or wrong) to different questions. However, the significant effect of learners' learning behaviors (e.g., answering speed, the number of attempts) is omitted, which can reflect their knowledge acquisition deeper and ensure the reliability of the response. In this paper, we propose a Learning Behavior-oriented Knowledge Tracing (LBKT) model, with the goal of explicitly exploring the learning behavior effects on learners' knowledge states. Specifically, we first analyze and summarize several dominated learning behaviors including Speed, Attempts and Hints in the learning process. As the characteristics of different learning behaviors vary greatly, we separately estimate their various effects on learners' knowledge acquisition in a quantitative manner. Then, considering that different learning behaviors are closely dependent with each other, we assess the fused effect of multiple learning behaviors by capturing their complex dependent patterns. Finally, we integrate the forgetting factor with learners' knowledge acquisition to comprehensively update their changing knowledge states in learning. Extensive experimental results on several public datasets demonstrate that our model generates better performance prediction for learners against existing methods. Moreover, LBKT shows good interpretability in tracking learners' knowledge state by incorporating the learning behavior effects. Our codes are available at https://github.com/xbh0720/LBKT.

Supplementary Material

MP4 File (rtfp0276-2min-promo.mp4)
We give a brief introduction to the importance of considering multiple learning behaviors' complex effects on assessing learners' knowledge states in knowledge tracing task and describe the workflow of our proposed Learning Behavior-oriented Knowledge Tracing (LBKT) model in explicitly quantifying the distinctive and cooperative effects of different behaviors.
MP4 File (rtfp0276-20min-video.mp4)
Presentation video for KDD'23 research paper "Learning Behavior-oriented Knowledge Tracing".

References

[1]
Solmaz Abdi, Hassan Khosravi, Shazia Sadiq, and Ali Darvishi. 2021. Open learner models for multi-activity educational systems. In Artificial Intelligence in Education: 22nd International Conference, AIED 2021, Utrecht, The Netherlands, June 14-18, 2021, Proceedings, Part II. Springer, 11--17.
[2]
Vincent Aleven and Kenneth R Koedinger. 2000. Limitations of student control: Do students know when they need help?. In International conference on intelligent tutoring systems. Springer, 292--303.
[3]
Ashton Anderson, Daniel Huttenlocher, Jon Kleinberg, and Jure Leskovec. 2014. Engaging with massive online courses. In Proceedings of the 23rd international conference on World wide web. 687--698.
[4]
Ryan Shaun Baker, Albert T Corbett, and Kenneth R Koedinger. 2004 a. Detecting student misuse of intelligent tutoring systems. In International conference on intelligent tutoring systems. Springer, 531--540.
[5]
Ryan S.J.d. Baker, Albert T. Corbett, Kenneth R. Koedinger, Shelley Evenson, Ido Roll, Angela Z. Wagner, Meghan Naim, Jay Raspat, Daniel J. Baker, and Joseph E. Beck. 2006. Adapting to when students game an intelligent tutoring system. In International conference on intelligent tutoring systems. Springer, 392--401.
[6]
Ryan Shaun Baker, Albert T Corbett, Kenneth R Koedinger, and Angela Z Wagner. 2004 b. Off-task behavior in the cognitive tutor classroom: When students" game the system". In Proceedings of the SIGCHI conference on Human factors in computing systems. 383--390.
[7]
Ryan Shaun Baker, Ido Roll, Albert T Corbett, and Kenneth R Koedinger. 2005. Do performance goals lead students to game the system?. In AIED. 57--64.
[8]
Tiffany Barnes. 2005. The Q-matrix method: Mining student response data for knowledge. In American association for artificial intelligence 2005 educational data mining workshop. AAAI Press, Pittsburgh, PA, USA, 1--8.
[9]
Penghe Chen, Yu Lu, Vincent W Zheng, and Yang Pian. 2018. Prerequisite-driven deep knowledge tracing. In 2018 IEEE International Conference on Data Mining (ICDM). IEEE, 39--48.
[10]
Ryan SJ d Baker, Sujith M Gowda, Michael Wixon, Jessica Kalka, Angela Z Wagner, Aatish Salvi, Vincent Aleven, Gail W Kusbit, Jaclyn Ocumpaugh, and Lisa Rossi. 2012. Towards Sensor-Free Affect Detection in Cognitive Tutor Algebra. International Educational Data Mining Society (2012).
[11]
Phung Do, Kha Nguyen, Thanh Nguyen Vu, Tran Nam Dung, and Tuan Dinh Le. 2017. Integrating knowledge-based reasoning algorithms and collaborative filtering into e-learning material recommendation system. In International Conference on Future Data and Security Engineering. Springer, 419--432.
[12]
John Domingue, Alexander Mikroyannidis, and Stefan Dietze. 2014. Online learning and linked data: lessons learned and best practices. In Proceedings of the 23rd International Conference on World Wide Web. 191--192.
[13]
Pragya Dwivedi, Vibhor Kant, and Kamal K Bharadwaj. 2018. Learning path recommendation based on modified variable length genetic algorithm. Education and information technologies, Vol. 23, 2 (2018), 819--836.
[14]
Susan E Embretson and Steven P Reise. 2013. Item response theory. Psychology Press.
[15]
Jean-Claude Falmagne, Eric Cosyn, Jean-Paul Doignon, and Nicolas Thiéry. 2006. The assessment of knowledge, in theory and in practice. In Formal Concept Analysis: 4th International Conference, ICFCA 2006, Dresden, Germany, February 13-17, 2006. Proceedings. Springer, 61--79.
[16]
Mingyu Feng, Neil Heffernan, and Kenneth Koedinger. 2009. Addressing the assessment challenge with an online system that tutors as it assesses. User modeling and user-adapted interaction, Vol. 19, 3 (2009), 243--266.
[17]
Aritra Ghosh, Neil Heffernan, and Andrew S Lan. 2020. Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining. 2330--2339.
[18]
Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, 249--256.
[19]
José González-Brenes, Yun Huang, and Peter Brusilovsky. 2014. General features in knowledge tracing to model multiple subskills, temporal item response theory, and expert knowledge. In The 7th international conference on educational data mining. University of Pittsburgh, 84--91.
[20]
Richard R Hake. 2002. Relationship of individual student normalized learning gains in mechanics with gender, high-school physics, and pretest scores on mathematics and spatial visualization. In Physics education research conference, Vol. 8. 1--14.
[21]
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, Vol. 9, 8 (1997), 1735--1780.
[22]
Zhenya Huang, Qi Liu, Yuying Chen, Le Wu, Keli Xiao, Enhong Chen, Haiping Ma, and Guoping Hu. 2020. Learning or forgetting? a dynamic approach for tracking the knowledge proficiency of students. ACM Transactions on Information Systems (TOIS), Vol. 38, 2 (2020), 1--33.
[23]
Jeffrey Johns and Beverly Woolf. 2006. A dynamic mixture model to detect student motivation and proficiency. In AAAI. 163--168.
[24]
E Joseph. 2005. Engagement tracing: using response times to model student disengagement. Artificial intelligence in education: Supporting learning through intelligent and socially informed technology, Vol. 125 (2005), 88.
[25]
Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[26]
Jiayu Liu, Zhenya Huang, Chengxiang Zhai, and Qi Liu. 2023 a. Learning by Applying: A General Framework for Mathematical Reasoning via Enhancing Explicit Knowledge Learning. arXiv preprint arXiv:2302.05717 (2023).
[27]
Qi Liu, Zhenya Huang, Yu Yin, Enhong Chen, Hui Xiong, Yu Su, and Guoping Hu. 2019. Ekt: Exercise-aware knowledge tracing for student performance prediction. IEEE Transactions on Knowledge and Data Engineering, Vol. 33, 1 (2019), 100--115.
[28]
Qi Liu, Shuanghong Shen, Zhenya Huang, Enhong Chen, and Yonghe Zheng. 2021. A survey of knowledge tracing. arXiv preprint arXiv:2105.15106 (2021).
[29]
Zitao Liu, Qiongqiong Liu, Jiahao Chen, Shuyan Huang, Boyu Gao, Weiqi Luo, and Jian Weng. 2023 b. Enhancing Deep Knowledge Tracing with Auxiliary Tasks. arXiv preprint arXiv:2302.07942 (2023).
[30]
Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. 2018. Efficient low-rank multimodal fusion with modality-specific factors. arXiv preprint arXiv:1806.00064 (2018).
[31]
Ting Long, Yunfei Liu, Jian Shen, Weinan Zhang, and Yong Yu. 2021. Tracing knowledge state with individual cognition and acquisition estimation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 173--182.
[32]
Kritphong Mongkhonvanit, Klint Kanopka, and David Lang. 2019. Deep knowledge tracing and engagement with moocs. In Proceedings of the 9th international conference on learning analytics & knowledge. 340--342.
[33]
Kasia Muldner, Winslow Burleson, Brett Van de Sande, and Kurt VanLehn. 2011. An analysis of students' gaming behaviors in an intelligent tutoring system: Predictors and impacts. User modeling and user-adapted interaction, Vol. 21, 1 (2011), 99--135.
[34]
Koki Nagatani, Qian Zhang, Masahiro Sato, Yan-Ying Chen, Francine Chen, and Tomoko Ohkuma. 2019. Augmenting knowledge tracing by considering forgetting behavior. In The world wide web conference. 3101--3107.
[35]
Hiromi Nakagawa, Yusuke Iwasawa, and Yutaka Matsuo. 2019. Graph-based knowledge tracing: modeling student proficiency using graph neural network. In 2019 IEEE/WIC/ACM International Conference On Web Intelligence (WI). IEEE, 156--163.
[36]
Shalini Pandey and George Karypis. 2019. A self-attentive model for knowledge tracing. arXiv preprint arXiv:1907.06837 (2019).
[37]
Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. Advances in neural information processing systems, Vol. 28 (2015).
[38]
Shi Pu and Lee Becker. 2022. Self-attention in Knowledge Tracing: Why It Works. In International Conference on Artificial Intelligence in Education. Springer, 731--736.
[39]
Sarah Schultz and Ivon Arroyo. 2014. Tracing knowledge and engagement in parallel in an intelligent tutoring system. In Educational Data Mining 2014.
[40]
Shuanghong Shen, Qi Liu, Enhong Chen, Zhenya Huang, Wei Huang, Yu Yin, Yu Su, and Shijin Wang. 2021. Learning process-consistent knowledge tracing. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1452--1460.
[41]
Shuanghong Shen, Qi Liu, Enhong Chen, Han Wu, Zhenya Huang, Weihao Zhao, Yu Su, Haiping Ma, and Shijin Wang. 2020. Convolutional knowledge tracing: Modeling individualization in student learning process. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. 1857--1860.
[42]
Xia Sun, Xu Zhao, Bo Li, Yuan Ma, Richard Sutcliffe, and Jun Feng. 2021. Dynamic key-value memory networks with rich features for knowledge tracing. IEEE Transactions on Cybernetics (2021).
[43]
Yue Suo, Naoki Miyata, Hiroki Morikawa, Toru Ishida, and Yuanchun Shi. 2008. Open smart classroom: Extensible and scalable learning system in smart space using web service technology. IEEE transactions on knowledge and data engineering, Vol. 21, 6 (2008), 814--828.
[44]
Nguyen Thai-Nghe, Lucas Drumond, Tomávs Horváth, Artus Krohn-Grimberghe, Alexandros Nanopoulos, and Lars Schmidt-Thieme. 2012. Factorization techniques for predicting student performance. In Educational recommender systems and technologies: Practices and challenges. IGI Global, 129--153.
[45]
Angel de Vicente and Helen Pain. 2002. Informing the detection of the students' motivational state: an empirical study. In International Conference on Intelligent Tutoring Systems. Springer, 933--943.
[46]
Chunpai Wang, Siqian Zhao, and Shaghayegh Sahebi. 2021. Learning from Non-Assessed Resources: Deep Multi-Type Knowledge Tracing. International Educational Data Mining Society (2021).
[47]
Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, Shijin Wang, and Yu Su. 2022. NeuralCD: A General Framework for Cognitive Diagnosis. IEEE Transactions on Knowledge and Data Engineering (2022).
[48]
Shuhan Wang, Hao Wu, Ji Hun Kim, and Erik Andersen. 2019. Adaptive learning material recommendation in online language education. In International conference on artificial intelligence in education. Springer, 298--302.
[49]
Yutao Wang and Neil T Heffernan. 2012. Leveraging First Response Time into the Knowledge Tracing Model. International Educational Data Mining Society (2012).
[50]
Runze Wu, Guandong Xu, Enhong Chen, Qi Liu, and Wan Ng. 2017. Knowledge or gaming? Cognitive modelling based on multiple-attempt response. In Proceedings of the 26th International Conference on World Wide Web Companion. 321--329.
[51]
Haiqin Yang and Lap Pong Cheung. 2018. Implicit heterogeneous features embedding in deep knowledge tracing. Cognitive Computation, Vol. 10, 1 (2018), 3--14.
[52]
Michael V Yudelson, Kenneth R Koedinger, and Geoffrey J Gordon. 2013. Individualized bayesian knowledge tracing models. In International conference on artificial intelligence in education. Springer, 171--180.
[53]
Amir Zadeh, Minghai Chen, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2017. Tensor fusion network for multimodal sentiment analysis. arXiv preprint arXiv:1707.07250 (2017).
[54]
Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017a. Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on World Wide Web. 765--774.
[55]
Liang Zhang, Xiaolu Xiong, Siyuan Zhao, Anthony Botelho, and Neil T Heffernan. 2017b. Incorporating rich features into deep knowledge tracing. In Proceedings of the fourth (2017) ACM conference on learning@ scale. 169--172.
[56]
Chuang Zhao, Hongke Zhao, Ming He, Jian Zhang, and Jianping Fan. 2023. Cross-domain recommendation via user interest alignment. arXiv preprint arXiv:2301.11467 (2023).
[57]
Siqian Zhao, Chunpai Wang, and Shaghayegh Sahebi. 2020. Modeling knowledge acquisition from multiple learning resource types. arXiv preprint arXiv:2006.13390 (2020).

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2023
5996 pages
ISBN:9798400701030
DOI:10.1145/3580305
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 August 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. knowledge tracing
  2. learning behaviors
  3. user modeling

Qualifiers

  • Research-article

Funding Sources

Conference

KDD '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)818
  • Downloads (Last 6 weeks)61
Reflects downloads up to 28 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media