Pull together: : Option-weighting-enhanced mixture-of-experts knowledge tracing
Authors: 
Tao Huang, Xinjia Ou, Huali Yang, Shengze Hu, Jing Geng, Zhuoran Xu, Zongkai YangAuthors Info & Claims
Tao Huang, Xinjia Ou, Huali Yang, Shengze Hu, Jing Geng, Zhuoran Xu, Zongkai YangAuthors Info & ClaimsReferences
[1]
An, S., Kim, J., Kim, M., & Park, J. (2022). No Task Left Behind: Multi-Task Learning of Knowledge Tracing and Option Tracing for Better Student Assessment. In Proceedings of the AAAI conference on artificial intelligence, vol. 36, no. 4 (pp. 4424–4431).
[2]
Baker R.S.d., Corbett A.T., Aleven V., More accurate student modeling through contextual estimation of slip and guess probabilities in bayesian knowledge tracing, in: Intelligent tutoring systems: 9th international conference, ITS 2008, montreal, Canada, June 23-27, 2008 proceedings 9, Springer, 2008, pp. 406–415.
[3]
Burton R.F., Misinformation, partial knowledge and guessing in true/false tests, Medical Education 36 (9) (2002) 805–811.
[4]
Chaudhry R., Singh H., Dogga P., Saini S.K., Modeling hint-taking behavior and knowledge state of students with multi-task learning., International Educational Data Mining Society (2018).
[5]
Choi, Y., Lee, Y., Cho, J., Baek, J., Kim, B., Cha, Y., et al. (2020). Towards an appropriate query, key, and value computation for knowledge tracing. In Proceedings of the seventh ACM conference on learning@ scale (pp. 341–344).
[6]
Corbett A.T., Anderson J.R., Knowledge tracing: Modeling the acquisition of procedural knowledge, User Modeling and User-Adapted Interaction 4 (1994) 253–278.
[7]
DeVietti T.L., Larson R.C., ECS effects: Evidence supporting state-dependent learning in rats, Journal of Comparative and Physiological Psychology 74 (3) (1971) 407.
[8]
Devin C., Gupta A., Darrell T., Abbeel P., Levine S., Learning modular neural network policies for multi-task and multi-robot transfer, in: 2017 IEEE international conference on robotics and automation, IEEE, 2017, pp. 2169–2176.
[9]
Diedenhofen B., Musch J., Empirical option weights improve the validity of a multiple-choice knowledge test, European Journal of Psychological Assessment (2015).
[10]
Diedenhofen B., Musch J., Option weights should be determined empirically and not by experts when assessing knowledge with multiple-choice items, International Journal of Selection and Assessment 27 (3) (2019) 256–266.
[11]
Fei, H., Zhang, J., Zhou, X., Zhao, J., Qi, X., & Li, P. (2021). GemNN: gating-enhanced multi-task neural networks with feature interaction learning for CTR prediction. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (pp. 2166–2171).
[12]
Frary R.B., Partial-credit scoring methods for multiple-choice tests, Applied Measurement in Education 2 (1) (1989) 79–96.
[13]
Ghosh, A., Heffernan, N., & Lan, A. S. (2020). Context-aware attentive knowledge tracing. In Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 2330–2339).
[14]
Ghosh A., Raspat J., Lan A., Option tracing: Beyond correctness analysis in knowledge tracing, in: Artificial intelligence in education: 22nd international conference, AIED 2021, Utrecht, the Netherlands, June 14–18, 2021, proceedings, part I, Springer, 2021, pp. 137–149.
[15]
Guo D., Hong J., Luo B., Yan Q., Niu Z., Multi-modal representation learning for short video understanding and recommendation, in: 2019 IEEE international conference on multimedia & expo workshops, IEEE, 2019, pp. 687–690.
[16]
Haladyna T.M., Effects of empirical option weighting on estimating domain scores and making pass/fail decisions, Applied Measurement in Education 3 (3) (1990) 231–244.
[17]
Huang T., Hu S., Yang H., Geng J., Li Z., Xu Z., et al., Response speed enhanced fine-grained knowledge tracing: A multi-task learning perspective, Expert Systems with Applications 238 (2024).
[18]
Huang T., She Q., Wang Z., Zhang J., GateNet: gating-enhanced deep network for click-through rate prediction, 2020, arXiv preprint arXiv:2007.03519.
[19]
Hutchinson T., Some theories of performance in multiple choice tests, and their implications for variants of the task, British Journal of Mathematical and Statistical Psychology 35 (1) (1982) 71–89.
[20]
Kasneci E., Seßler K., Küchemann S., Bannert M., Dementieva D., Fischer F., et al., ChatGPT for good? On opportunities and challenges of large language models for education, Learning and individual differences 103 (2023).
[21]
Kendall, A., Gal, Y., & Cipolla, R. (2018). Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7482–7491).
[22]
Lau P.N.K., Lau S.H., Hong K.S., Usop H., Guessing, partial knowledge, and misconceptions in multiple-choice tests, Journal of Educational Technology & Society 14 (4) (2011) 99–110.
[23]
Lee, W., Chun, J., Lee, Y., Park, K., & Park, S. (2022). Contrastive learning for knowledge tracing. In Proceedings of the ACM web conference 2022 (pp. 2330–2338).
[24]
Lee, S., Kim, K. S., Shin, J., & Park, J. (2021). Tracing Knowledge for Tracing Dropouts: Multi-Task Training for Study Session Dropout Prediction. In EDM.
[25]
Lin C., Chi M., Intervention-bkt: incorporating instructional interventions into bayesian knowledge tracing, in: Intelligent tutoring systems: 13th international conference, ITS 2016, zagreb, Croatia, June 7-10, 2016. proceedings 13, Springer, 2016, pp. 208–218.
[26]
Liu, Z., Liu, Q., Chen, J., Huang, S., Gao, B., Luo, W., et al. (2023). Enhancing deep knowledge tracing with auxiliary tasks. In Proceedings of the ACM web conference 2023 (pp. 4178–4187).
[27]
Liu S., Yu J., Li Q., Liang R., Zhang Y., Shen X., et al., Ability boosted knowledge tracing, Information Sciences 596 (2022) 567–587.
[28]
Lord F.M., Formula scoring and number-right scoring, Journal of Educational Measurement (1975) 7–11.
[29]
Luo, Y., Zhou, H., Tu, W.-W., Chen, Y., Dai, W., & Yang, Q. (2020). Network on network for tabular data classification in real-world applications. In Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (pp. 2317–2326).
[30]
Ma, J., Zhao, Z., Yi, X., Chen, J., Hong, L., & Chi, E. H. (2018). Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining (pp. 1930–1939).
[31]
Nakagawa, H., Iwasawa, Y., & Matsuo, Y. (2019). Graph-based knowledge tracing: modeling student proficiency using graph neural network. In IEEE/WIC/aCM international conference on web intelligence (pp. 156–163).
[32]
Overton D.A., Statedependent or” dissociated” learning produced with pentobarbital, Journal of Comparative and Physiological Psychology 57 (1) (1964) 3.
[33]
Pandey S., Karypis G., A self-attentive model for knowledge tracing, 2019, arXiv preprint arXiv:1907.06837.
[34]
Pardos Z.A., Heffernan N.T., KT-IDEM: Introducing item difficulty to the knowledge tracing model, in: User modeling, adaption and personalization: 19th international conference, UMAP 2011, Girona, Spain, July 11-15, 2011. proceedings 19, Springer, 2011, pp. 243–254.
[35]
Piech C., Bassen J., Huang J., Ganguli S., Sahami M., Guibas L.J., et al., Deep knowledge tracing, Advances in Neural Information Processing Systems 28 (2015).
[36]
Pinto L., Gupta A., Learning to push by grasping: Using multiple tasks for effective learning, in: 2017 IEEE international conference on robotics and automation, IEEE, 2017, pp. 2161–2168.
[37]
Pramanik S., Agrawal P., Hussain A., Omninet: A unified architecture for multi-modal multi-task learning, 2019, arXiv preprint arXiv:1907.07804.
[38]
Rasheed R.A., Kamsin A., Abdullah N.A., Challenges in the online component of blended learning: A systematic review, Computers & Education 144 (2020).
[39]
Shen, S., Huang, Z., Liu, Q., Su, Y., Wang, S., & Chen, E. (2022). Assessing Student’s Dynamic Knowledge State by Exploring the Question Difficulty Effect. In Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval (pp. 427–437).
[40]
Shen, S., Liu, Q., Chen, E., Huang, Z., Huang, W., Yin, Y., et al. (2021). Learning process-consistent knowledge tracing. In Proceedings of the 27th ACM SIGKDD conference on knowledge discovery & data mining (pp. 1452–1460).
[41]
Shin, D., Shim, Y., Yu, H., Lee, S., Kim, B., & Choi, Y. (2021). Saint+: Integrating temporal features for ednet correctness prediction. In LAK21: 11th international learning analytics and knowledge conference (pp. 490–496).
[42]
Su, Y., Liu, Q., Liu, Q., Huang, Z., Yin, Y., Chen, E., et al. (2018). Exercise-enhanced sequential modeling for student performance prediction. In Proceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1.
[43]
Tang, H., Liu, J., Zhao, M., & Gong, X. (2020). Progressive layered extraction (ple): A novel multi-task learning (mtl) model for personalized recommendations. In Proceedings of the 14th ACM conference on recommender systems (pp. 269–278).
[44]
Vandenhende S., Georgoulis S., Van Gool L., Mti-net: Multi-scale task interaction networks for multi-task learning, in: Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part IV 16, Springer, 2020, pp. 527–543.
[45]
Wang, C., Ma, W., Zhang, M., Lv, C., Wan, F., Lin, H., et al. (2021). Temporal cross-effects in knowledge tracing. In Proceedings of the 14th ACM international conference on web search and data mining (pp. 517–525).
[46]
Wang W., Ma H., Zhao Y., Li Z., He X., Tracking knowledge proficiency of students with calibrated Q-matrix, Expert Systems with Applications 192 (2022).
[47]
Wang W., Ma H., Zhao Y., Yang F., Chang L., SEEP: Semantic-enhanced question embeddings pre-training for improving knowledge tracing, Information Sciences 614 (2022) 153–169.
[48]
Watfrs B.K., The measurement of partial knowledge a comparison between two empirical option-weighting methods and rights-only scoring, The Journal of Educational Research 69 (7) (1976) 256–260.
[49]
Weinstein C.E., Mayer R.E., The teaching of learning strategies, in: Innovation abstracts, 5, ERIC, 1983, p. n32.
[50]
Wu Z., Huang L., Huang Q., Huang C., Tang Y., SGKT: Session graph-based knowledge tracing for student performance prediction, Expert Systems with Applications 206 (2022).
[51]
Wu T., Ling Q., Self-supervised heterogeneous hypergraph network for knowledge tracing, Information Sciences 624 (2023) 200–216.
[52]
Xu, B., Huang, Z., Liu, J., Shen, S., Liu, Q., Chen, E., et al. (2023). Learning Behavior-oriented Knowledge Tracing. In Proceedings of the 29th ACM SIGKDD conference on knowledge discovery and data mining (pp. 2789–2800).
[53]
Yang H., Hu S., Geng J., Huang T., Hu J., Zhang H., et al., Heterogeneous graph-based knowledge tracing with spatiotemporal evolution, Expert Systems with Applications (2023).
[54]
Yudelson M.V., Koedinger K.R., Gordon G.J., Individualized bayesian knowledge tracing models, in: Artificial intelligence in education: 16th international conference, AIED 2013, Memphis, TN, USA, July 9-13, 2013. proceedings 16, Springer, 2013, pp. 171–180.
[55]
Zhang, J., Shi, X., King, I., & Yeung, D.-Y. (2017). Dynamic key-value memory networks for knowledge tracing. In Proceedings of the 26th international conference on world wide web (pp. 765–774).
[56]
Zhang Y., Yang Q., A survey on multi-task learning, IEEE Transactions on Knowledge and Data Engineering 34 (12) (2021) 5586–5609.
[57]
Zhang K., Yao Y., A three learning states Bayesian knowledge tracing model, Knowledge-Based Systems 148 (2018) 189–201.
[58]
Zhao Y., Ma H., Wang W., Gao W., Yang F., He X., Exploiting multiple question factors for knowledge tracing, Expert Systems with Applications 223 (2023).
[59]
Zheng L., Wang C., Chen X., Song Y., Meng Z., Zhang R., Evolutionary machine learning builds smart education big data platform: Data-driven higher education, Applied Soft Computing 136 (2023).
Recommendations
Knowledge-Enhanced Multi-task Learning for Course Recommendation
Database Systems for Advanced ApplicationsAbstractKnowledge tracing (KT) aims to model learners’ knowledge level and predict future performance given their past interactions in learning applications. Adaptive learning systems mainly generate course recommendations based on learner’s knowledge ...
Response speed enhanced fine-grained knowledge tracing: A multi-task learning perspective
AbstractThe primary objective of knowledge tracing (KT) is to trace learners’ changing knowledge states and predict their future performance by analyzing their learning trajectories. One of the fundamental assumptions underpinning KT is that estimating ...
Highlights- We propose two steps of consistency between performance and knowledge state in KT.
- We add an extra response speed prediction task for KT with multi-task learning.
- We design and optimize an encoder–decoder–predictor framework for ...
Structured Knowledge Tracing Models for Student Assessment on Coursera
L@S '16: Proceedings of the Third (2016) ACM Conference on Learning @ ScaleMassive Open Online Courses (MOOCs) provide an effective learning platform with various high-quality educational materials accessible to learners from all over the world. However, current MOOCs lack personalized learning guidance and intelligent ...
Comments
Information & Contributors
Information
Published In
Elsevier Ltd.
Publisher
Pergamon Press, Inc.
United States
Publication History
Published: 17 July 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025