Large language models (LLMs) can generate source code quickly but often mis-interpret free-text instructions. We study whether supplying a formal input—UML class diagrams—reduces these errors and how prompt design influences the outcome. Three LLMs (GPT-4o, Gemini 1.5 Pro and DeepSeek Reasoner) are asked to turn four diagrams of increasing complexity into Java 11 code. Each diagram is processed twice: with a short baseline prompt and with a detailed expert prompt that lists coding rules and project constraints. We measure structural accuracy, compilation-error rate and generation time over 20 iterations per condition. With the expert prompt, Gemini reaches 95% structural accuracy and 94% successful compilations, outperforming GPT-4o (31%, 28%) and DeepSeek (46%, 34%). GPT-4o is the fastest generator (median 160 s), while DeepSeek is an order of magnitude slower. The expert prompt raises accuracy by eight percentage points on average and halves the error rate, yet adds no significant latency. Performance drops for all models as diagram complexity grows. These results show that this preliminary study demonstrates that model-driven prompts can make LLM code generation more reliable and that prompt quality may matter as much as model choice.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Prompt-Guided Evaluation of GPT-4o, Gemini and DeepSeek on UML-to-Java Code Generation

  • Miguel Escobar,
  • Rene Noel,
  • Oscar Pastor

摘要

Large language models (LLMs) can generate source code quickly but often mis-interpret free-text instructions. We study whether supplying a formal input—UML class diagrams—reduces these errors and how prompt design influences the outcome. Three LLMs (GPT-4o, Gemini 1.5 Pro and DeepSeek Reasoner) are asked to turn four diagrams of increasing complexity into Java 11 code. Each diagram is processed twice: with a short baseline prompt and with a detailed expert prompt that lists coding rules and project constraints. We measure structural accuracy, compilation-error rate and generation time over 20 iterations per condition. With the expert prompt, Gemini reaches 95% structural accuracy and 94% successful compilations, outperforming GPT-4o (31%, 28%) and DeepSeek (46%, 34%). GPT-4o is the fastest generator (median 160 s), while DeepSeek is an order of magnitude slower. The expert prompt raises accuracy by eight percentage points on average and halves the error rate, yet adds no significant latency. Performance drops for all models as diagram complexity grows. These results show that this preliminary study demonstrates that model-driven prompts can make LLM code generation more reliable and that prompt quality may matter as much as model choice.