RAG-KT: Retrieval Augmented Generation Based Difficulty Estimation for Knowledge Tracing

Knowledge Tracing (KT) is a key task in online education that aims to model the dynamic knowledge states of students. To achieve high-quality KT, question difficulty estimation plays an important role during the information extraction phase. However, most existing methods based on statistical models face two major challenges: (1) unstable estimates across different datasets, and (2) the cold-start problem when estimating the difficulty of new questions. To address these issues, we propose a Retrieval-Augmented Generation-based Knowledge Tracing framework (RAG-KT). This framework introduces a novel RAG-based difficulty estimation method that generates stable and accurate difficulty scores, and can estimate the difficulty of unseen questions by referencing semantically similar ones. The LLM-generated scores, along with other relevant information from students’ learning histories, are then fed into a KT model for the prediction task. Specifically, to improve the quality of difficulty estimation, we apply contrastive learning to fine-tune a domain-specific embedding model for RAG in educational contexts. We also carefully design task-specific prompts to guide LLMs for better difficulty estimation performance. Extensive experimental results demonstrate that RAG-KT achieves stable difficulty estimation and effectively addresses the cold-start problem. With more reliable difficulty estimates, the performance of KT models can be significantly improved.