The unified difficulty ranking mechanism for automatic multiple choice question generation in digital storytelling domain

Multiple Choice Questions (MCQs) are an important evaluation technique for both examinations and learning activities. However, the manual creation of questions is time-consuming and challenging for teachers. Hence, there is a notable demand for an Automatic Question Generation (AQG) system. Several systems have been created for this aim, but the generated questions failed to meet the requirements for student assessment effectively. Consequently, research in education technology, natural language processing, and KG development technology to support the AQG systems is still in its infancy. In this paper, the innovative integrated AQG framework for creating MCQs with difficulty levels is presented. The improved KG is first built as a source for question generation through corresponding queries. The distractors or wrong choices are generated by proposing a unified difficulty ranking mechanism, which includes the hybrid technique of WordNet-based and Linked Data (LD)-based semantic similarity together with property filtering score. Furthermore, the syntactic feature i.e. part-of-speech is utilized for the best distractors generation. The experimental results of the proposed unified difficulty ranking mechanism demonstrate an accuracy of 94% on the KG test dataset and 75% on public datasets. The accuracy of distractors’ correctness is 60% on the test dataset while it is 72% on the public dataset. The results highlight the efficiency of the proposed unified difficulty ranking mechanism for automatic MCQs generation in the digital storytelling domain.