Scholarlink

<p>This study examines the effectiveness of large language models (LLMs) in generating automated educational feedback for open-ended written questions without a single correct answer. Two LLMs–GPT-4o-mini and Llama-3.2 11B–were used to produce feedback on responses to open-ended questions from a high-stakes situational judgment test. Both base (prompted) and fine-tuned versions of the models were evaluated. Fine-tuning was conducted using a dataset containing human-crafted feedback examples. The quality of model-generated feedback was assessed using a standardized rubric and compared across models with automatic similarity metrics. Results reveal notable differences in the feedback outputted by open-source (Llama) and closed-source (GPT) systems, as well as performance variations before and after fine-tuning. However, fine-tuning did not yield a clear advantage over high-quality prompting, suggesting that prompt design may rival fine-tuning in producing effective automated feedback.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Is it worth the effort? a comparison of automatic educational feedback generated by base and fine-tuned LLMs

  • 期刊论文

  • 发表日期：

    2026年4月20日

查看全文
  • Elisabetta Mazzullo,
  • Okan Bulut,
  • Daniel Jerez,
  • Kevin Vo,
  • Cole Walsh,
  • Gill Sitarenios,
  • Alexander MacIntosh

摘要

This study examines the effectiveness of large language models (LLMs) in generating automated educational feedback for open-ended written questions without a single correct answer. Two LLMs–GPT-4o-mini and Llama-3.2 11B–were used to produce feedback on responses to open-ended questions from a high-stakes situational judgment test. Both base (prompted) and fine-tuned versions of the models were evaluated. Fine-tuning was conducted using a dataset containing human-crafted feedback examples. The quality of model-generated feedback was assessed using a standardized rubric and compared across models with automatic similarity metrics. Results reveal notable differences in the feedback outputted by open-source (Llama) and closed-source (GPT) systems, as well as performance variations before and after fine-tuning. However, fine-tuning did not yield a clear advantage over high-quality prompting, suggesting that prompt design may rival fine-tuning in producing effective automated feedback.