Mitigating Spurious Correlations in Few-Shot Classification via Bias and Dynamic Prompt
摘要
Vision-language models (VLMs) are renowned for their extensive prior knowledge and strong generalization abilities. However, in few-shot tasks, the limited number of training samples often leads to knowledge collapse, exacerbating the model’s reliance on spurious correlations. Existing methods for few-shot classification typically do not address this issue, particularly in complex environments or with challenging data distributions. To overcome these challenges, we propose the Bias and Dynamic Prompt Contrastive Language-Image Pre-Training (BDPC) model. Our approach introduces an Attention Bias Module (ABM) to improve the model's understanding of limited samples, effectively bridging the semantic gap between images and text, and enhancing robustness to spurious correlations. Additionally, we incorporate a Dynamic Interval Prompt (DIP) to adaptively adjust the model’s behavior, mitigating the knowledge collapse issue and further improving performance. Extensive experiments across 11 diverse datasets demonstrate the effectiveness of our method.