De-Noising Tail Entity Selection in Automatic Question Generation with Fine-Tuned T5 Model

Automatic question generation is the task of producing questions from a given text passage, with neural approaches currently achieving state-of-the-art results. While generating questions, selection of tail entity (named entity in the later part of the question) is a critical aspect, and neural network models require improvement in the process. In addition, most exam question generation approaches can generate questions that are too general without tail entities, where lack of specificity can end up with redundant questions. To address the challenge, proposed work aims to learn a subject domain-constrained dependency parser using a Spider Monkey Optimizer for tail entity extraction. The extracted tail entities are ranked by relatedness using a Composite Lexical Similarity Metric (CLSM). Finally, a T5 transformer is fine-tuned using the generated entity as a parameter for the question generation task. The proposed approach is evaluated on two datasets, each with 1k and 2k sentences, respectively, and the results are compared against existing baseline models. Our experimental results show that the proposed fine-tuned T5-based question generation approach outperforms other techniques over existing baselines in terms of ROUGE-2, ROUGE-L, and WER.