In images composed of foreground and background regions, the background often contains redundant information. We have observed that the degree of background redundancy influences image classification model performance. However, existing image cropping techniques primarily focus on composition optimization for aesthetic purposes to enhance visual comfort and are not tailored for specific downstream tasks. Therefore, we propose a novel foreground-background semantic-aware image augmentation method called CropAug. This method dynamically crops redundant backgrounds based on the analysis of foreground-background similarity and performs context-aware mixing. Specifically, we first obtain the foreground mask through precise segmentation and then extract deep semantic features from both the foreground and background regions. By calculating the cosine similarity between these features, we design an activation function to map the semantic relevance to pixel-level cropping points. Finally, the cropped and retained backgrounds are adaptively mixed. This approach allows for the removal of low-relevance redundant backgrounds while retaining key contextual information for classification and enables the model to better learn foreground details. Experimental results on multiple datasets demonstrate that our method outperforms existing state-of-the-art (SOTA) techniques.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

CropAug: Controllable Region Cropping for Fine-Grained Data Augmentation

  • Mengyao Zhou,
  • Shuo Wang,
  • Yanmin Chen,
  • Jun Luo

摘要

In images composed of foreground and background regions, the background often contains redundant information. We have observed that the degree of background redundancy influences image classification model performance. However, existing image cropping techniques primarily focus on composition optimization for aesthetic purposes to enhance visual comfort and are not tailored for specific downstream tasks. Therefore, we propose a novel foreground-background semantic-aware image augmentation method called CropAug. This method dynamically crops redundant backgrounds based on the analysis of foreground-background similarity and performs context-aware mixing. Specifically, we first obtain the foreground mask through precise segmentation and then extract deep semantic features from both the foreground and background regions. By calculating the cosine similarity between these features, we design an activation function to map the semantic relevance to pixel-level cropping points. Finally, the cropped and retained backgrounds are adaptively mixed. This approach allows for the removal of low-relevance redundant backgrounds while retaining key contextual information for classification and enables the model to better learn foreground details. Experimental results on multiple datasets demonstrate that our method outperforms existing state-of-the-art (SOTA) techniques.