Retry and Refine: A Multi-agent Framework for LLM Based Reliable Unit Test Generation
摘要
Automatic unit test generation remains a persistent challenge in software engineering, particularly as codebases grow in complexity. This paper introduces a multi agent framework for LLM based unit test generation that emulates the developer driven refinement cycle through structured retries, validation, and feedback guided regeneration. Built using the LangGraph orchestration framework, the system integrates seven specialized agents, each handling a specific task such as user story generation, validation, test construction, and execution. Evaluations on MBPP and HumanEval benchmarks with GPT-4o-mini and DeepSeek-v3 reveal substantial improvements, with Pass@3 success rates increasing by up to 50% points compared to baseline pipelines. An ablation study shows the impact of incorporating expected output feedback into the regeneration task, showing reduced retry depth and improved convergence. The framework also captures retry depth distributions and test coverage trade offs, offering diagnostic insight into LLM behavior. These results demonstrate the benefits of agentic orchestration and feedback guided regeneration in building robust systems for LLM based software unit testing.