Automated Grading in Coding Exercises Using Large Language Models

In recent years, Massive Open Online Courses (MOOCs) have gained widespread popularity as a means to provide flexible and accessible education to a diverse range of learners. However, MOOCs face a significant challenge in the form of the laborious and time-consuming task of grading, particularly for coding assignments. Prior research has shown the potential of language models in several natural language processing tasks, including code generation and commenting. However, their utility in grading coding exercises within learning environments like MOOCs remains an area that requires further exploration. Additionally, the effectiveness of such models in less-common languages, such as Greek, has not been thoroughly investigated. This study aims to evaluate the efficacy of Large Language Models (LLMs), particularly GPT-3.5 and GPT-4, in automatically grading code submissions in a Greek MOOC setting, focusing on an introductory Python course.