Self-Supervised Conjoint CLIP for Unsupervised Domain Adaptation

Traditional unsupervised domain adaptation (UDA) methods often suffer from two critical limitations: lack of class discriminability (LCD) and class-level semantic ignorance (CSI), resulting from coarse grained feature alignment strategies. Recently, vision-language models (VLMs) such as CLIP have shown promise by exploring cross-modal semantic information; however, current CLIP-based prompt learning approaches fail to fully capture fine-grained visual patterns, resulting in a Semantic-Visual Granularity Gap (SVGG). To address the problem, we propose Self-supervised Conjoint CLIP (SCC), a dual-branch architecture that integrates CLIP’s semantic alignment capabilities with fine grained self-supervised visual representations from DINOv2. Specifically, SCC comprises a CLIP base branch optimized via multi-modal prompt learning to preserve cross-modal semantic alignment, and a DINOv2 enhanced branch incorporating a novel DINOv2-CLIP Mutual Tuning (DCMT) module. The DCMT module employs cross-attention mechanisms guided by the Alignment-Uniformity principle: cross-attention aligns semantic and visual representations via subspace alignment, while a self-consistency loss ensures uniformity. Extensive experiments on three benchmarks demonstrate that SCC improves domain adaptability and outperforms other methods.