Object detection, a crucial task in computer vision, has undergone remarkable advancements with deep learning technologies. Deep learning-based object detection models have gained significant attention due to their powerful performance. However, when these models are deployed in real-world environments, the images they receive may be affected by various factors, leading to irreparable losses. Current research focuses on examining the robustness of individual models while giving limited consideration to a holistic evaluation of robustness over a wide range of models. Therefore, we propose multi-level critical transformation robustness (CTR) metrics to quantitatively evaluating the sensitivity of models in object detection, and introduce a robustness evaluation method to assess detection performance across a set of diverse input conditions. We present a comparative performance analysis of various models under standardized test conditions and their adaptability to environmental changes such as lighting, perspective, and image quality. The evaluation results demonstrate that object detection models with different deep learning architectures, even if they have comparable parameter sizes and predictive capabilities, exhibit significant differences in robustness when handling perturbations caused by environmental changes. These findings can guide the selection of model structures for practical implementations. Additionally, a method is proposed based on our robustness metrics to assist selection of data augmentation techniques and their parameter intervals during retraining.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multi-level Critical Transformation Robustness Evaluation of Object Detection Models

  • Zhen Zhang,
  • Chengye Li,
  • Zheheng Liang,
  • Chaosheng Yao,
  • Jinbo Zhang,
  • Rongjie Yan,
  • Peng Wu

摘要

Object detection, a crucial task in computer vision, has undergone remarkable advancements with deep learning technologies. Deep learning-based object detection models have gained significant attention due to their powerful performance. However, when these models are deployed in real-world environments, the images they receive may be affected by various factors, leading to irreparable losses. Current research focuses on examining the robustness of individual models while giving limited consideration to a holistic evaluation of robustness over a wide range of models. Therefore, we propose multi-level critical transformation robustness (CTR) metrics to quantitatively evaluating the sensitivity of models in object detection, and introduce a robustness evaluation method to assess detection performance across a set of diverse input conditions. We present a comparative performance analysis of various models under standardized test conditions and their adaptability to environmental changes such as lighting, perspective, and image quality. The evaluation results demonstrate that object detection models with different deep learning architectures, even if they have comparable parameter sizes and predictive capabilities, exhibit significant differences in robustness when handling perturbations caused by environmental changes. These findings can guide the selection of model structures for practical implementations. Additionally, a method is proposed based on our robustness metrics to assist selection of data augmentation techniques and their parameter intervals during retraining.