Scholarlink

<p>Prompt learning is an effective approach for adapting pre-trained vision-language models (VLMs) to a variety of downstream tasks. However, prompts designed manually or generated by large language models may not effectively capture key discriminative visual features. In addition, pre-trained VLMs may not align images and text well at a fine-grained level. To address these two issues, we propose an attention-enhanced cross-modality alignment network, which includes an adaptive channel attention (ACA) module and a cross-modal measurement (CMM) module. The ACA module adapts the existing efficient channel attention to highlight discriminative visual and textual features. The CMM module leverages four pairs of image-text similarities across both frozen and learnable branches, improving the alignment of fine-grained discriminative visual and textual features. Experiments show that the proposed method outperforms state-of-the-art methods on two representative tasks: base-to-novel generalization and cross-dataset evaluation. Our code is available at <a href="https://github.com/xueshaoying/XSY_AECA.git">https://github.com/xueshaoying/XSY_AECA.git</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Attention-Enhanced Cross-Modality Alignment for Adapting Vision-Language Models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月10日

查看全文
  • Shaoying Xue,
  • Xiaochen Yang,
  • Xiaoxu Li,
  • Jie Cao,
  • Zhanyu Ma

摘要

Prompt learning is an effective approach for adapting pre-trained vision-language models (VLMs) to a variety of downstream tasks. However, prompts designed manually or generated by large language models may not effectively capture key discriminative visual features. In addition, pre-trained VLMs may not align images and text well at a fine-grained level. To address these two issues, we propose an attention-enhanced cross-modality alignment network, which includes an adaptive channel attention (ACA) module and a cross-modal measurement (CMM) module. The ACA module adapts the existing efficient channel attention to highlight discriminative visual and textual features. The CMM module leverages four pairs of image-text similarities across both frozen and learnable branches, improving the alignment of fine-grained discriminative visual and textual features. Experiments show that the proposed method outperforms state-of-the-art methods on two representative tasks: base-to-novel generalization and cross-dataset evaluation. Our code is available at https://github.com/xueshaoying/XSY_AECA.git.