In this study aims to provide a methodology for generating source code based on natural language prompts and analyze the impact of detailed development documentation on the quality of software constructed using the generated source code. Using three types of prompts that progressively reflect a requirements specification, a basic design document, and a detailed design document, source code was generated by an LLM, and 39 tests were conducted in the form of use cases. As a result, a trend was observed in which the quality of the software improved as the level of detail reflected in the prompt increased. Furthermore, it was found that for very small-scale software, most functions could be implemented by presenting only the functional requirements. However, the need for users to explicitly specify items that do not pertain to functional compatibility, such as exception handling, and the necessity of measures to address the increase in input token count were pointed out.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Investigating the Relationship Between Quality and Prompt Specificity in Source Code Generation by LLM

  • Leon Nomoto,
  • Hironori Takuma

摘要

In this study aims to provide a methodology for generating source code based on natural language prompts and analyze the impact of detailed development documentation on the quality of software constructed using the generated source code. Using three types of prompts that progressively reflect a requirements specification, a basic design document, and a detailed design document, source code was generated by an LLM, and 39 tests were conducted in the form of use cases. As a result, a trend was observed in which the quality of the software improved as the level of detail reflected in the prompt increased. Furthermore, it was found that for very small-scale software, most functions could be implemented by presenting only the functional requirements. However, the need for users to explicitly specify items that do not pertain to functional compatibility, such as exception handling, and the necessity of measures to address the increase in input token count were pointed out.