skip to main content
10.1145/3534678.3539399acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Incremental Cognitive Diagnosis for Intelligent Education

Published: 14 August 2022 Publication History

Abstract

Cognitive diagnosis, aiming at providing an approach to reveal the proficiency level of learners on knowledge concepts, plays an important role in intelligent education area and has recently received more and more attention. Although a number of works have been proposed in recent years, most of contemporary works acquire the traits parameters of learners and items in a transductive way, which are only suitable for stationary data. However, in the real scenario, the data is collected online, where learners, test items and interactions usually grow continuously, which can rarely meet the stationary condition. To this end, we propose a novel framework, Incremental Cognitive Diagnosis (ICD), to tailor cognitive diagnosis into the online scenario of intelligent education. Specifically, we first design a Deep Trait Network (DTN), which acquires the trait parameters in an inductive way rather than a transductive way. Then, we propose an Incremental Update Algorithm (IUA) to balance the effectiveness and training efficiency. We carry out Turning Point (TP) analysis to reduce update frequency, where we derive the minimum update condition based on the monotonicity theory of cognitive diagnosis. Meanwhile, we use a momentum update strategy on the incremental data to decrease update time without sacrificing effectiveness. Moreover, to keep the trait parameters as stable as possible, we refine the loss function in the incremental updating stage. Last but no least, our ICD is a general framework which can be applied to most of contemporary cognitive diagnosis models. To the best of our knowledge, this is the first attempt to investigate the incremental cognitive diagnosis problem with theoretical results about the update condition and a tailored incremental learning strategy. Extensive experiments demonstrate the effectiveness and robustness of our method.

References

[1]
A. Barreto, D. Precup, and J. Pineau. On-line reinforcement learning using incremental kernel-based stochastic factorization. Advances in Neural Information Processing Systems, 25:1484--1492, 2012.
[2]
S. Chen and T. Joachims. Predicting matchups and preferences in context. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 775--784, 2016.
[3]
Y. Chen, X. Li, J. Liu, and Z. Ying. Recommendation system for adaptive learning. Applied psychological measurement, 42(1):24--41, 2018.
[4]
J. De La Torre. Dina model and parameter estimation: A didactic. Journal of educational and behavioral statistics, 34(1):115--130, 2009.
[5]
R. Devooght, N. Kourtellis, and A. Mantrach. Dynamic matrix factorization with priors on unknown values. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 189--198, 2015.
[6]
E. Diaz-Aviles, L. Drumond, L. Schmidt-Thieme, and W. Nejdl. Real-time top-n recommendation in social streams. In Proceedings of the sixth ACM conference on Recommender systems, pages 59--66, 2012.
[7]
A. Gepperth and B. Hammer. Incremental learning algorithms and applications. In European symposium on artificial neural networks (ESANN), 2016.
[8]
X. Glorot and Y. Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics, pages 249--256, 2010.
[9]
K. He, H. Fan, Y.Wu, S. Xie, and R. Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9729--9738, 2020.
[10]
X. Huang, L.Wu, E. Chen, H. Zhu, Q. Liu, Y.Wang, and B. T. I. Center. Incremental matrix factorization: A linear feature transformation perspective. In IJCAI, pages 1901--1908, 2017.
[11]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[12]
Y.-W. Lee and Y. Sawaki. Cognitive diagnosis approaches to language assessment: An overview. Language Assessment Quarterly, 6(3):172--189, 2009.
[13]
Q. Liu. Towards a new generation of cognitive diagnosis. In Z. Zhou, editor, Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021, Virtual Event / Montreal, Canada, 19--27 August 2021, pages 4961--4964. ijcai.org, 2021.
[14]
Q. Liu, S. Tong, C. Liu, H. Zhao, E. Chen, H. Ma, and S.Wang. Exploiting cognitive structure for adaptive learning. In A. Teredesai, V. Kumar, Y. Li, R. Rosales, E. Terzi, and G. Karypis, editors, Proceedings of the 25th ACMSIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2019, Anchorage, AK, USA, August 4--8, 2019, pages 627--635. ACM, 2019.
[15]
Q. Liu, R.Wu, E. Chen, G. Xu, Y. Su, Z. Chen, and G. Hu. Fuzzy cognitive diagnosis for modelling examinee performance. ACM Transactions on Intelligent Systems and Technology (TIST), 9(4):1--26, 2018.
[16]
F. Lord. A theory of test scores. Psychometric monographs, 1952.
[17]
F. M. Lord. Applications of item response theory to practical testing problems. Routledge, 1980.
[18]
X. Luo, Y. Xia, and Q. Zhu. Incremental collaborative filtering recommender based on regularized matrix factorization. Knowledge-Based Systems, 27:271--280, 2012.
[19]
L. v. d. Maaten and G. Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579--2605, 2008.
[20]
Z. Mai, R. Li, J. Jeong, D. Quispe, H. Kim, and S. Sanner. Online continual learning in image classification: An empirical survey. Neurocomputing, 469:28--51, 2022.
[21]
P. Matuszyk, J. Vinagre, M. Spiliopoulou, A. M. Jorge, and J. Gama. Forgetting methods for incremental matrix factorization in recommender systems. In Proceedings of the 30th Annual ACM Symposium on Applied Computing, pages 947--953, 2015.
[22]
M. D. Reckase. Multidimensional item response theory models. In Multidimensional item response theory, pages 79--112. Springer, 2009.
[23]
S. Rendle and L. Schmidt-Thieme. Online-updating regularized kernel matrix factorization models for large-scale recommender systems. In Proceedings of the 2008 ACM conference on Recommender systems, pages 251--258, 2008.
[24]
P. R. Rosenbaum. Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49(3):425--435, 1984.
[25]
B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. Incremental singular value decomposition algorithms for highly scalable recommender systems. In Fifth international conference on computer and information science, volume 1, pages 27--8. Citeseer, 2002.
[26]
Q. Song, J. Cheng, and H. Lu. Incremental matrix factorization via feature space re-learning for recommender system. In Proceedings of the 9th ACM Conference on Recommender Systems, pages 277--280, 2015.
[27]
M. A. Sorrel, J. Olea, F. J. Abad, J. de la Torre, D. Aguado, and F. Lievens. Validity and reliability of situational judgement test scores: A new approach based on cognitive diagnosis models. Organizational Research Methods, 19(3):506--532, 2016.
[28]
G. Takács, I. Pilászy, B. Németh, and D. Tikk. Scalable collaborative filtering approaches for large recommender systems. The Journal of Machine Learning Research, 10:623--656, 2009.
[29]
K. K. Tatsuoka. Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. Cognitively diagnostic assessment, pages 327--359, 1995.
[30]
N. Thai-Nghe, T. Horváth, and L. Schmidt-Thieme. Factorization models for forecasting student performance. In Educational Data Mining 2011. Citeseer, 2010.
[31]
S. Tong, Q. Liu, R. Yu,W. Huang, Z. Huang, Z. Pardos, andW. Jiang. Item response ranking for cognitive diagnosis. IJCAI, 2021.
[32]
E. Tsutsumi, R. Kinoshita, and M. Ueno. Deep-irt with independent student and item networks. International Educational Data Mining Society, 2021.
[33]
J. Vinagre, A. M. Jorge, and J. Gama. Fast incremental matrix factorization for recommendation with positive-only feedback. In International Conference on User Modeling, Adaptation, and Personalization, pages 459--470. Springer, 2014.
[34]
F. Wang, Q. Liu, E. Chen, Z. Huang, Y. Chen, Y. Yin, Z. Huang, and S. Wang. Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6153--6161, 2020.
[35]
F. Wang, H. Tong, and C. Y. Lin. Towards evolutionary nonnegative matrix factorization. In 25th AAAI Conference on Artificial Intelligence and the 23rd Innovative Applications of Artificial Intelligence Conference, AAAI-11/IAAI-11, pages 501--506, 2011.
[36]
M.Wu, R. L. Davis, B.W. Domingue, C. Piech, and N. Goodman. Variational item response theory: Fast, accurate, and expressive. arXiv preprint arXiv:2002.00276, 2020.
[37]
J. Xu, C. Deng, X. Gao, D. Shen, and H. Huang. Predicting alzheimer's disease cognitive assessment via robust low-rank structured sparse model. In IJCAI: proceedings of the conference, volume 2017, page 3880. NIH Public Access, 2017.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '22: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
August 2022
5033 pages
ISBN:9781450393850
DOI:10.1145/3534678
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 August 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cognitive diagnosis
  2. incremental learning
  3. inductive learning
  4. transductive learning

Qualifiers

  • Research-article

Funding Sources

  • the National Key Research and Development Program of China
  • the National Natural Science Foundation of China
  • the Young Elite Scientists Sponsorship Program by CAST and ISZS

Conference

KDD '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)154
  • Downloads (Last 6 weeks)21
Reflects downloads up to 27 Jan 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media