Scholarlink

<p>Image-text matching is a pivotal task in multimodal research, aiming to establish fine-grained semantic associations between visual and textual data for accurate cross-modal similarity estimation. Traditional approaches often struggle with complex and ambiguous correspondences, primarily due to overlooking semantic continuity in text and high appearance resemblance in visually similar objects. To address these challenges, we propose a novel method, Enhancing Image-Text Matching through Contextual Fine-Grained Alignment (CEFG), which enhances semantic alignment accuracy through dual-modal context modeling. In the visual branch, we introduce the Spatial Context-Aware Module (SCAM), which models spatial relationships between image regions to improve understanding of object attributes and their semantic roles. In the textual branch, we propose the Hierarchical Semantic Parsing Network (HSPN) to extract Abstract Meaning Representation (AMR) from sentences, identifying object categories and uncovering implicit inter-object relationships. Our experiments on the Flickr30K and MS-COCO datasets demonstrate that CEFG consistently outperforms state-of-the-art methods, achieving rSum gains of 10.2% and 2.1%, respectively. Here, we show that our method significantly enhances cross-modal matching performance by effectively capturing and utilizing contextual information. Our code is available at <a href="https://github.com/MMM-design/CEFG">https://github.com/MMM-design/CEFG</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing image-text matching through contextual fine-grained alignment

  • 期刊论文

  • 发表日期：

    2025年12月15日

查看全文
  • FanRong Meng,
  • Dezhi Han,
  • Xiang Shen,
  • Chongqing Chen,
  • Qun Wang

摘要

Image-text matching is a pivotal task in multimodal research, aiming to establish fine-grained semantic associations between visual and textual data for accurate cross-modal similarity estimation. Traditional approaches often struggle with complex and ambiguous correspondences, primarily due to overlooking semantic continuity in text and high appearance resemblance in visually similar objects. To address these challenges, we propose a novel method, Enhancing Image-Text Matching through Contextual Fine-Grained Alignment (CEFG), which enhances semantic alignment accuracy through dual-modal context modeling. In the visual branch, we introduce the Spatial Context-Aware Module (SCAM), which models spatial relationships between image regions to improve understanding of object attributes and their semantic roles. In the textual branch, we propose the Hierarchical Semantic Parsing Network (HSPN) to extract Abstract Meaning Representation (AMR) from sentences, identifying object categories and uncovering implicit inter-object relationships. Our experiments on the Flickr30K and MS-COCO datasets demonstrate that CEFG consistently outperforms state-of-the-art methods, achieving rSum gains of 10.2% and 2.1%, respectively. Here, we show that our method significantly enhances cross-modal matching performance by effectively capturing and utilizing contextual information. Our code is available at https://github.com/MMM-design/CEFG.