Code generation has long been a challenging task in natural language processing, with existing models often struggling to produce correct and functional code solutions. This paper explores integrating Behavior-Driven Development (BDD)—a user-centric agile methodology—into the code generation process. We propose BDDCoder, a novel multi-agent framework comprising four roles: Programmer, Tester, Requirements Analyst, and User, designed to simulate real-world BDD workflows. BDDCoder consists of two variants: BDD-NL, which uses natural language scenarios for code generation and LLM-based self-validation and BDD-Test, which converts scenarios into executable test cases for code validation. Through empirical evaluation on benchmark datasets (HumanEval, MBPP, and their EvalPlus variants), we demonstrate that BDD-NL with LLM self-validation could hinder code generation performance, while BDD-Test significantly outperforms BDD-NL, achieving up to a 15.1% improvement in Pass@1 scores. Our findings highlight the potential of BDD to enhance requirement clarity and code alignment with user needs, offering a robust framework for future research on integrating software engineering methodologies into automated code generation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Exploring Behavior-Driven Development for Code Generation

  • Yunhao Liang,
  • Chengguang Gan,
  • Ruixuan Ying,
  • Zhe Cui

摘要

Code generation has long been a challenging task in natural language processing, with existing models often struggling to produce correct and functional code solutions. This paper explores integrating Behavior-Driven Development (BDD)—a user-centric agile methodology—into the code generation process. We propose BDDCoder, a novel multi-agent framework comprising four roles: Programmer, Tester, Requirements Analyst, and User, designed to simulate real-world BDD workflows. BDDCoder consists of two variants: BDD-NL, which uses natural language scenarios for code generation and LLM-based self-validation and BDD-Test, which converts scenarios into executable test cases for code validation. Through empirical evaluation on benchmark datasets (HumanEval, MBPP, and their EvalPlus variants), we demonstrate that BDD-NL with LLM self-validation could hinder code generation performance, while BDD-Test significantly outperforms BDD-NL, achieving up to a 15.1% improvement in Pass@1 scores. Our findings highlight the potential of BDD to enhance requirement clarity and code alignment with user needs, offering a robust framework for future research on integrating software engineering methodologies into automated code generation.