Self-supervised Co-salient Object Detection via Unified Multi-granularity Feature Learning
摘要
Co-salient object detection (CoSOD) seeks to identify common salient objects across related image groups, traditionally relying on supervised learning with extensive manual annotations. To address this challenge, we propose UMS-CoSOD, a novel self-supervised framework that eliminates the need for labeled data while achieving state-of-the-art performance. Our approach unfolds in three stages: cross-modal pseudo-label generation, feature learning and saliency prediction, and result refinement and boundary enhancement (RRBE). Initially, high-quality pseudo-labels are generated via a cross-modal self-supervised strategy, integrating instance segmentation masks and self-attention maps from different modalities of self-supervised information. The core feature learning stage employs an Adaptive Feature Mixer Module (AFMM) for intelligent fusion of initial features, a Comprehensive Feature Mining Module (CFMM) to capture inter-image co-salient cues, and a Feature Refinement and Amplification Module (FRAM) for feature enhancement. During inference, Adaptive Threshold Binarization, Region-Level Feature Refinement, and Dense CRFs are used to obtain correct and clearly bounded results. Evaluated on benchmark datasets (CoCA, CoSOD3k and CoSal2015), UMS-CoSOD significantly outperforms unsupervised methods, reducing MAE by 19.8% on CoSOD3k. It also achieves comparable performance to top-tier fully supervised methods. This framework offers a practical, annotation-free solution for CoSOD, with broad applicability in real-world scenarios. The code is available at UMS-CoSOD .