Using Knowledge Graphs to Improve Question Difficulty Estimation from Text

Question Difficulty Estimation (QDE) is a crucial task in many educational settings. Previous research focused on Natural Language Processing (NLP) to overcome the limitations of traditional QDE methods, but no work experimented the use of Knowledge Graphs (KGs) to provide a taxonomy of the topics assessed in exams. We propose two ways of incorporating KG information into existing models for QDE from text and, by experimenting on a publicly available dataset, show that they outperform the models that use text information exclusively, with a decrease in MAE of up to 8% with respect to the best-performing baseline (BERT-based QDE). We study how the models generalise to topics different from those used for training, and observe that while in most cases KGs are still capable of outperforming the baselines, a simpler model such as DistilBERT is more robust to previously unseen topics.