A Unified Conceptual Hybrid Framework for the Automated Assessment of Short Answers
摘要
Various automated essay scoring (AES) methods have been proposed over the past five decades, but the application of AES in the educational field has gained popularity since the COVID-19 pandemic, as most educational institutions have shifted to online teaching modes. Consequently, the assessment of student knowledge has become a major challenge. Researchers are focusing on new state-of-the-art techniques to devise a more performant AES to facilitate online grading tasks. However, few studies have analyzed the common features of existing AES. There are no general guiding principles for the implementation and improvement of AES. This work aims to address the research gap by proposing a unified conceptual hybrid framework for AES, adapted for short answers and inspired by an in-depth analysis of existing AES based on short answers. The unified framework consists mainly of the most frequently used components in existing AES except that a new important module namely the data augmentation module has been identified and added to the framework and also two existing modules have been modified. Following the proposed unified framework, the different essential components in an AES can be easily understood, and the implementation and improvement of AES can be achieved effortlessly using a hybrid approach. Furthermore, experiments have been carried out to validate the framework’s performance. During the experimentation, single, ensemble, and hybrid models were compared with and without the data augmentation technique respectively. Results confirmed that the data augmentation module did help in improving results for all the models. Notably, in the hybrid models, results demonstrated an average increase of 40.75% in QWK values and a mean reduction of 18% in RMSE values. Additionally, ensemble and hybrid models outperformed single models in terms of performance.