Automated assessment tools (AATs) are widely used in computer science education to grade programming assignments at scale. However, many existing tools focus solely on correctness, overlooking crucial aspects such as code readability, elegance, and documentation. This study explores the use of ChatGPT-4o for a more holistic evaluation, assessing multiple dimensions of student submissions while generating detailed feedback. To enhance feedback quality, we integrate a Human-in-the-Loop (HIL) process, where human reviewers refine ChatGPT-4o’s feedback, enabling iterative improvements. We evaluate the model’s performance under zero-shot, one-shot, and five-shot settings, demonstrating how increased contextual information improves grading consistency and feedback accuracy. Experimental results show that this hybrid approach significantly enhances feedback structure, clarity, and usability. Our findings suggest that combining LLMs with human oversight can improve grading efficiency, provide students with actionable insights, and make AI-assisted grading a viable solution for programming education. This study highlights the potential of integrating LLMs with human expertise to create a scalable and effective automated grading system.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing ChatGPT-4o's Automated Programming Assignment Grading and Feedback with Human-in-the-Loop Refinement

  • Ruoyu Xu,
  • Gaoxiang Li,
  • Victor S. Sheng

摘要

Automated assessment tools (AATs) are widely used in computer science education to grade programming assignments at scale. However, many existing tools focus solely on correctness, overlooking crucial aspects such as code readability, elegance, and documentation. This study explores the use of ChatGPT-4o for a more holistic evaluation, assessing multiple dimensions of student submissions while generating detailed feedback. To enhance feedback quality, we integrate a Human-in-the-Loop (HIL) process, where human reviewers refine ChatGPT-4o’s feedback, enabling iterative improvements. We evaluate the model’s performance under zero-shot, one-shot, and five-shot settings, demonstrating how increased contextual information improves grading consistency and feedback accuracy. Experimental results show that this hybrid approach significantly enhances feedback structure, clarity, and usability. Our findings suggest that combining LLMs with human oversight can improve grading efficiency, provide students with actionable insights, and make AI-assisted grading a viable solution for programming education. This study highlights the potential of integrating LLMs with human expertise to create a scalable and effective automated grading system.