Development of Automated Essay Scoring System Using DeBERTa as a Transformer-Based Language Model

Giving an essay assignment is an important task every educational institution holds for measuring students’ understanding and ability. Teachers have a significant role in this task because they are the one who assesses the assignment. Over time, the number of students will increase too. This makes doing a manual correction become more complicated. Some downsides are taking too much time, being less objective, and many more. In the past few years, the problem of automated essay scoring has become popular. But this problem still has a big problem which makes automated essay scoring still not as good as human assessment, especially in detecting the main idea of an essay, cohesion, and coherence. The large language model (LLM) has become popular in the past few years. Some examples are transformer-based models such as BERT, RoBERTa, and DeBERTa. In this research, we implement those three models as our base layer, in which we compare the value of Quadratic Weighted Kappa (QWK) as the metric of accuracy. In conclusion, the DeBERTa-based model has the best value of QWK compared to the other two. We also implement a system using Python, that can retrieve an essay and will run the model to do the scoring automatically. We also suggest further research references that can use different datasets other than the ASAP-AES dataset for the research or try to use the GPT-based language model as the base layer model.