Exploring quality criteria and evaluation methods in automated question generation: A comprehensive survey

In light of the widespread adoption of technology-enhanced learning and assessment platforms, there is a growing demand for innovative, high-quality, and diverse assessment questions. Automatic Question Generation (AQG) has emerged as a valuable solution, enabling educators and assessment developers to efficiently produce a large volume of test items, questions, or assessments within a short timeframe. AQG leverages computer algorithms to automatically generate questions, streamlining the question-generation process. Despite the efficiency gains, significant gaps in the question-generation pipeline hinder the seamless integration of AQG systems into the assessment process. Notably, the absence of a standardized evaluation framework poses a substantial challenge in assessing the quality and usability of automatically generated questions. This study addresses this gap by conducting a comprehensive survey of existing question evaluation methods, a crucial step in refining the question generation pipeline. Subsequently, we present a taxonomy for these evaluation methods, shedding light on their respective advantages and limitations within the AQG context. The study concludes by offering recommendations for future research to enhance the effectiveness of AQG systems in educational assessments.