GUI automation testing is a mainstream approach to ensure the software quality of mobile applications. To reduce manual testing costs, a large number of automated test cases are typically executed for regression and compatibility testing whenever there are requirement changes or version updates. Although extensive research has applied LLMs and MLLMs to GUI automation, most of these works conduct testing in stable, interference-free environments. In contrast, real-world business scenarios often involve numerous dynamic interference factors and strong business-specific contexts, leading to lower success rates for these methods. To address these challenges, we propose a novel UI automation testing technology (UI-Most) based on a multi-agent architecture. This method is designed to enhance the robustness of UI automation testing by assigning specialized roles to independent agents and enabling their collaboration. At the same time, it leverages the business knowledge from AppGraph for one-shot learning, thereby improving the recognition of UI elements in new scenarios. The effectiveness of our approach has been validated on real test case sets. Furthermore, this method has already been applied to automated regression testing, significantly reducing both manual testing costs and maintenance overhead of test cases.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

UI-Most: Leveraging Multi-agent Systems for One-Shot Automatic GUI Testing

  • Wenlong Du,
  • Ruichen Li,
  • Jingfei Yu,
  • Feng Guo,
  • Boshi Li,
  • Jing Wang

摘要

GUI automation testing is a mainstream approach to ensure the software quality of mobile applications. To reduce manual testing costs, a large number of automated test cases are typically executed for regression and compatibility testing whenever there are requirement changes or version updates. Although extensive research has applied LLMs and MLLMs to GUI automation, most of these works conduct testing in stable, interference-free environments. In contrast, real-world business scenarios often involve numerous dynamic interference factors and strong business-specific contexts, leading to lower success rates for these methods. To address these challenges, we propose a novel UI automation testing technology (UI-Most) based on a multi-agent architecture. This method is designed to enhance the robustness of UI automation testing by assigning specialized roles to independent agents and enabling their collaboration. At the same time, it leverages the business knowledge from AppGraph for one-shot learning, thereby improving the recognition of UI elements in new scenarios. The effectiveness of our approach has been validated on real test case sets. Furthermore, this method has already been applied to automated regression testing, significantly reducing both manual testing costs and maintenance overhead of test cases.