skip to main content
10.1145/3583131.3590522acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Probabilistic model with evolutionary optimization for cognitive diagnosis

Published: 12 July 2023 Publication History

Abstract

Cognitive Diagnostic Models (CDMs) aim to analyze students' cognitive levels of each knowledge component (KC) by mining educational data. Existing CDMs can be mainly divided into two categories, i.e., traditional probability-based and neural-network-based. Most probabilistic models have the advantages of simplicity and good interpretability, but suffer from slow training time in the case of a large number of KCs. Neural-network-based methods are widely considered to be superior to probabilistic models due to their good performance. However, neural network methods are less interpretable than probabilistic models, thus limiting their usefulness in practice. Because most existing probabilistic models are optimized iteratively based on single-point-based search methods, they may be easily trapped in local optimum due to the influence of the initial points. And evolutionary algorithms (EAs) have good global search ability. Therefore, an interesting question is whether a simple probabilistic model based on evolutionary optimization can rival neural-network in limited optimization time. Thus, a hybrid EA with a customized local search is proposed. Experimental results on three real-world datasets show that our method outperforms the compared 7 models (including 2 state-of-the-art neural-network-based models); and the running time of our method is significantly less than the compared probabilistic models.

References

[1]
Cristobal Romero and Sebastian Ventura. Educational data mining and learning analytics: An updated survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 10(3):e1355, 2020.
[2]
Behdad Bakhshinategh, Osmar R Zaiane, Samira ElAtia, and Donald Ipperciel. Educational data mining applications and tasks: A survey of the last 10 years. Education and Information Technologies, 23:537--553, 2018.
[3]
Said A Salloum, Muhammad Alshurideh, Ashraf Elnagar, and Khaled Shaalan. Mining in educational data: review and future directions. In Proceedings of the International Conference on Artificial Intelligence and Computer Vision (AICV2020), pages 92--102. Springer, 2020.
[4]
Xuegang Hu, Fei Liu, and Chenyang Bu. Research advances on knowledge tracing models in educational big data. Journal of Computer Research and Development, 57(12):2523--2546, 2020.
[5]
Yingjie Liu, Tiancheng Zhang, Xuecen Wang, Ge Yu, and Tao Li. New development of cognitive diagnosis models. Frontiers of Computer Science, 17(1):171604, 2023.
[6]
Yunxiao Chen, Xiaoou Li, Jingchen Liu, and Zhiliang Ying. Recommendation system for adaptive learning. Applied psychological measurement, 42(1):24--41, 2018.
[7]
Zhengyang Wu, Ming Li, Yong Tang, and Qingyu Liang. Exercise recommendation based on knowledge concept prediction. Knowledge-Based Systems, 210:106481, 2020.
[8]
Yujia Huo, Derek F Wong, Lionel M Ni, Lidia S Chao, and Jing Zhang. Knowledge modeling via contextualized representations for lstm-based personalized exercise recommendation. Information Sciences, 523:266--278, 2020.
[9]
Rui Luo, Fei Liu, Wenhao Liang, Yuhong Zhang, Chenyang Bu, and Xuegang Hu. Dagkt: Difficulty and attempts boosted graph-based knowledge tracing. In Neural Information Processing: 29th International Conference, ICONIP 2022, Virtual Event, November 22--26, 2022, Proceedings, Part II, pages 255--266. Springer, 2023.
[10]
Qi Liu. Towards a new generation of cognitive diagnosis. In Proceedings of the Thirtieth International foint Conference on Artificial Intelligence, IJCAI-21., pages 4961--4964, 2021.
[11]
Jimmy de la Torre. Dina model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34(1):115--130, 2009.
[12]
Louis V DiBello, Louis A Roussos, and William Stout. 31a review of cognitively diagnostic assessment and a summary of psychometric models. Handbook of statistics, 26:979--1030, 2006.
[13]
Chenyang Bu, Fei Liu, Zhiyong Cao, Lei Li, Yuhong Zhang, Xuegang Hu, and Wenjian Luo. Cognitive diagnostic model made more practical by genetic algorithm. IEEE Transactions on Emerging Topics in Computational Intelligence, 7(2):447--461, 2022.
[14]
Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yuying Chen, Yu Yin, Zai Huang, and Shijin Wang. Neural cognitive diagnosis for intelligent education systems. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 6153--6161, 2020.
[15]
Fei Wang, Qi Liu, Enhong Chen, Zhenya Huang, Yu Yin, Shijin Wang, and Yu Su. NeuralCD: A general framework for cognitive diagnosis. IEEE Transactions on Knowledge & Data Engineering, 037(01):1--16, 2022.
[16]
Fei Liu, Xuegang Hu, Chenyang Bu, and Kui Yu. Fuzzy bayesian knowledge tracing. IEEE Transactions on Fuzzy Systems, 30(7):2412--2425, 2022.
[17]
Mark D. Reckase. Multidimensional Item Response Theory Models, pages 79--112. Springer New York, 2009.
[18]
Seyedali Mirjalili. Evolutionary algorithms and neural networks. In Studies in computational intelligence, volume 780. Springer, 2019.
[19]
Susan E Embretson and Steven P Reise. Item response theory. Psychology Press, 2013.
[20]
Chun Wang, Hua-Hua Chang, and Jeffery Douglas. Combining cat with cognitive diagnosis: A weighted item selection approach. Behavior Research Methods, 44(1):95--109, 2012.
[21]
Nathan Minchen and Jimmy de la Torre. A general cognitive diagnosis model for continuous-response data. Measurement: Interdisciplinary Research and Perspectives, 16(1):30--44, 2018.
[22]
Lu Dong, Zhenhua Ling, Qiang Ling, and Zefeng Lai. Cognitive diagnosis with explicit student vector estimation and unsupervised question matrix learning. arXiv e-prints, pages arXiv-2203, 2022.
[23]
Chun-Kit Yeung. Deep-IRT: Make deep learning based knowledge tracing explainable using item response theory. arXiv preprint arXiv:1904.11738, 2019.
[24]
Weibo Gao, Qi Liu, Zhenya Huang, Yu Yin, Haoyang Bi, Mu-Chun Wang, Jianhui Ma, Shijin Wang, and Yu Su. RCD: Relation map driven cognitive diagnosis for intelligent education systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 501--510, New York, NY, USA, 2021. Association for Computing Machinery.
[25]
Tianlong Qi, Meirui Ren, Longjiang Guo, Xiaokun Li, Jin Li, and Lichen Zhang. ICD: A new interpretable cognitive diagnosis model for intelligent tutor systems. Expert Systems with Applications, 215:119309, 2023.
[26]
Ling Ling Wang, Tao Xin, and Liu Yanlou. An improved parameter-estimating method in bayesian networks applied for cognitive diagnosis assessment. Frontiers in psychology, 12:665441, 2021.
[27]
Jimmy De La Torre and Jeffrey A Douglas. Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69(3):333--353, 2004.
[28]
Jonathan L Templin and Robert A Henson. Measurement of psychological disorders using cognitive diagnosis models. Psychological methods, 11(3):287, 2006.
[29]
Zhehan Jiang and Richard Carter. Using hamiltonian monte carlo to estimate the log-linear cognitive diagnosis model via stan. Behavior Research Methods, 51:651--662, 2019.
[30]
Lokman Akbay and Jimmy de la Torre. Estimation approaches in cognitive diagnosis modeling when attributes are hierarchically structured. 2020.
[31]
Dongbo Tu, Shiyu Wang, Yan Cai, Jeff Douglas, and Hua-Hua Chang. Cognitive diagnostic models with attribute hierarchies: Model estimation with a restricted q-matrix design. Applied psychological measurement, 43(4):255--271, 2019.
[32]
Yanlou Liu, Tao Xin, Björn Andersson, and Wei Tian. Information matrix estimation procedures for cognitive diagnostic models. British Journal of Mathematical and Statistical Psychology, 72(1):18--37, 2019.
[33]
Chao Wang, Qi Liu, Enhong Chen, Zhenya Huang, Tianyu Zhu, Yu Su, and Guoping Hu. The rapid calculation method of dina model for large scale cognitive diagnosis. Acta Electonica Sinica, 46(5):1047--10555, 2018.
[34]
Xin Lin, Wenjian Luo, and Peilan Xu. Differential evolution for multimodal optimization with species by nearest-better clustering. IEEE Transactions on Cybernetics, 51(2):970--983, 2019.
[35]
Chenyang Bu, Wenjian Luo, Tao Zhu, Ruikang Yi, and Bin Yang. Species and memory enhanced differential evolution for optimal power flow under double-sided uncertainties. IEEE Transactions on Sustainable Computing, 5(3):403--415, 2020.
[36]
Brian W Junker and Klaas Sijtsma. Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3):258--272, 2001.
[37]
Lawrence T DeCarlo. On the analysis of fraction subtraction data: The dina model, classification, latent class sizes, and the q-matrix. Applied Psychological Measurement, 35(1):8--26, 2011.
[38]
Qi Liu, Runze Wu, Enhong Chen, Guandong Xu, Yu Su, Zhigang Chen, and Guoping Hu. Fuzzy cognitive diagnosis for modelling examinee performance. ACM Transactions on Intelligent Systems and Technology (TIST), 9(4):1--26, 2018.
[39]
Tobias Blickle. Tournament selection. Evolutionary Computation, 1:181--186, 2000.
[40]
Gilbert Syswerda. Uniform crossover in genetic algorithms. In Proceedings of International Conference on Genetic Algorithms, volume 3, pages 2--9, 1989.
[41]
R. Hinterding. Gaussian mutation and self-adaption for numeric genetic algorithms. In Proceedings of 1995 IEEE International Conference on Evolutionary Computation, volume 1, pages 384--, 1995.
[42]
Andrew P Bradley. The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognition, 30(7):1145--1159, 1997.
[43]
Hongbin Pei, Bo Yang, Jiming Liu, and Lei Dong. Group sparse bayesian learning for active surveillance on epidemic dynamics. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
[44]
Tianfeng Chai and Roland R Draxler. Root mean square error (rmse) or mean absolute error (mae). Geoscientific Model Development Discussions, 7(1):1525--1534, 2014.
[45]
Tzu-Tsung Wong and Nai-Yu Yang. Dependency analysis of accuracy estimates in k-fold cross validation. IEEE Transactions on Knowledge and Data Engineering, 29(11):2417--2427, 2017.

Cited By

View all

Index Terms

  1. Probabilistic model with evolutionary optimization for cognitive diagnosis

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '23: Proceedings of the Genetic and Evolutionary Computation Conference
    July 2023
    1667 pages
    ISBN:9798400701191
    DOI:10.1145/3583131
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 July 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. educational data mining
    2. cognitive diagnostic
    3. evolutionary algorithm
    4. local search

    Qualifiers

    • Research-article

    Funding Sources

    • the National Natural Science Foundation of China
    • the Fundamental Research Funds for the Central Universities
    • the Anhui Provincial Natural Science Foundation

    Conference

    GECCO '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 102
      Total Downloads
    • Downloads (Last 12 months)42
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 25 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media