Quantifying the Reproducibility of Knowledge Tracing Models
摘要
The field of knowledge tracking (KT) has experienced substantial growth since the emergence of the first deep learning-based KT model in 2015. Many models utilizing deep learning approaches have emerged, asserting improvements in performance and prediction capacities. However, a systematic review of the literature on KT revealed that numerous models are hindered by inadequate or poorly constructed documentation, complicating the verification and reproduction of reported outcomes. This study seeks to tackle the significant issue of reproducibility in KT models by introducing a reproducibility evaluation approach designed exclusively for this field. Based on insights from reproducibility research in machine learning, it was found that reproducibility is not a binary concept, but rather occurs on a continuum, contingent upon the quality and thoroughness of documentation and experimental transparency. This research utilizes the deep knowledge tracking (DKT) model as a case study to illustrate the actual implementation of the proposed solution. This research offers a systematic methodology to evaluate the reproducibility of KT models, thus providing more dependable and verifiable research results in the discipline. This research promotes a culture of transparency and rigor in knowledge translation and related fields, guaranteeing that future discoveries are based on reproducible and reliable procedures.