Detecting icons in Graphical User Interfaces (GUIs) is essential for effective application automation. This study examines the impact of different annotation methods on the performance of object detection models for icon detection in GUIs. We compared manual, automated, and hybrid annotations using three models: Faster R-CNN, YOLOv8, and YOLOv9. The results show that manual annotations achieve the highest accuracy, with YOLOv9 reaching an Average Precision (AP) of 68.23% and Faster R-CNN achieving 61.82%. Hybrid methods that combine automated annotations with manual corrections also show significant improvements, though they do not perform as well as manual annotations alone. These findings underscore the importance of high-quality, consistent annotations for training effective detection models. While we used HTML code for automated annotations to simplify the process, we encountered inconsistencies that affected model performance. This highlights the need to develop better hybrid methods tailored to specific tasks, ensuring efficiency and accuracy in data annotation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The Impact of Data Annotations on the Performance of Object Detection Models in Icon Detection for GUI Images

  • Mădălina Dicu,
  • Enol García González,
  • Camelia Chira,
  • José R. Villar

摘要

Detecting icons in Graphical User Interfaces (GUIs) is essential for effective application automation. This study examines the impact of different annotation methods on the performance of object detection models for icon detection in GUIs. We compared manual, automated, and hybrid annotations using three models: Faster R-CNN, YOLOv8, and YOLOv9. The results show that manual annotations achieve the highest accuracy, with YOLOv9 reaching an Average Precision (AP) of 68.23% and Faster R-CNN achieving 61.82%. Hybrid methods that combine automated annotations with manual corrections also show significant improvements, though they do not perform as well as manual annotations alone. These findings underscore the importance of high-quality, consistent annotations for training effective detection models. While we used HTML code for automated annotations to simplify the process, we encountered inconsistencies that affected model performance. This highlights the need to develop better hybrid methods tailored to specific tasks, ensuring efficiency and accuracy in data annotation.