A Strategy for Implementing Domain-Based Task Generation and Evaluation System Using Text-Text Generative Models

In both traditional and fully online teaching models, the pedagogical metric for evaluating learning engagement is centered around exercise questions that assess the level of knowledge that learners have acquired with respect to a particular learning engagement. While it is impossible to run out of questions, manually generating contextually relevant and unique questions in a declarative programming course based on structured query language is an inevitable challenge. In our particular situation, we developed an automatic question generation system using the text-to-text transfer transformer and a corresponding evaluation system using the bidirectional encoder representations from transformers. To evaluate the usefulness and effectiveness of our implementation, we evaluated our system against the GPT-3.5 performance baseline. The performance of our prototype reaches a precision value of 0.44 and an Fβ=0.5 value of 0.42, while the Gpt-3.5 gives us 0.4 and 0.45 respective scores, which justifies our strategy of employing large language models for question/answer generation and evaluation.