Scholarlink

We describe a programming problem generator powered by OpenAI’s GPT-4o (or more recently, o1) that was used by an introductory programming class to prepare for midterms. The system had 59 unique student users out of a class of 192. We found that the system was much better at producing problems with solutions that were in the scope of the course material when it used o1 as the underlying LLM instead of GPT-4o; the model change raised the percentage of in-scope problems from 47.5 to 85%. The students generated rather more problems than expected, with an average of 16 or 17 problems per student per exam. However, there was no evidence that student grades on their midterms improved as a result of interacting with the problem generator.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

LLM Practice Problem Generation in a Beginning Programming Class

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Kevin Gold

摘要

We describe a programming problem generator powered by OpenAI’s GPT-4o (or more recently, o1) that was used by an introductory programming class to prepare for midterms. The system had 59 unique student users out of a class of 192. We found that the system was much better at producing problems with solutions that were in the scope of the course material when it used o1 as the underlying LLM instead of GPT-4o; the model change raised the percentage of in-scope problems from 47.5 to 85%. The students generated rather more problems than expected, with an average of 16 or 17 problems per student per exam. However, there was no evidence that student grades on their midterms improved as a result of interacting with the problem generator.