The advancement of Large Language Model (LLMs) has opened new possibilities for automating software development, particularly in generating test codes. This study examines LLM capabilities for autonomously generating Robot Framework test codes using a Retrieval-Augmented Generation (RAG) model. By leveraging a repository with stored contextual information, the model enhances the accuracy and relevance of generated test scripts while reducing human intervention. To assess the generated tests, we employ CodeBLEU to evaluate syntactic correctness, Jaccard Similarity to measure structural alignment, and pass/fail statistics from CI/CD pipelines to determine real-world applicability. Our experimental results demonstrate that integrating RAG with LLMs significantly improves both test quality and reliability. Contextual information retrieval enhances semantic cohesion and functional robustness, while a multi-metric evaluation framework ensures a comprehensive assessment. This study highlights the transformational role of LLMs in software testing automation, emphasizing their potential to streamline continuous integration (CI) practices. By enhancing test code generation accuracy and minimizing manual effort, the proposed approach marks a significant step toward fully automated and contextually aware testing within the Robot Framework.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Generating Robot Framework Code with LLM Models: A RAG-Based Approach

  • Anıl Sezgin,
  • Gürkan Özkan,
  • Esra Coşgun

摘要

The advancement of Large Language Model (LLMs) has opened new possibilities for automating software development, particularly in generating test codes. This study examines LLM capabilities for autonomously generating Robot Framework test codes using a Retrieval-Augmented Generation (RAG) model. By leveraging a repository with stored contextual information, the model enhances the accuracy and relevance of generated test scripts while reducing human intervention. To assess the generated tests, we employ CodeBLEU to evaluate syntactic correctness, Jaccard Similarity to measure structural alignment, and pass/fail statistics from CI/CD pipelines to determine real-world applicability. Our experimental results demonstrate that integrating RAG with LLMs significantly improves both test quality and reliability. Contextual information retrieval enhances semantic cohesion and functional robustness, while a multi-metric evaluation framework ensures a comprehensive assessment. This study highlights the transformational role of LLMs in software testing automation, emphasizing their potential to streamline continuous integration (CI) practices. By enhancing test code generation accuracy and minimizing manual effort, the proposed approach marks a significant step toward fully automated and contextually aware testing within the Robot Framework.