Scholarlink

<p>To address the limitations observed in existing partial multimodal hashing methods, including the insufficient representation capability of multimodal features and hash codes as well as the substantial bias in modality completion under high missing rates, this paper proposes a novel approach, termed Partial Multimodal Hashing with Multi-level Semantics and Adversarial Learning. The proposed framework first establishes a semantic-enhanced multimodal feature modeling scheme by jointly exploiting the CLIP and BLIP-2 models for feature extraction, where image-derived textual semantics are further generated to enrich textual semantic modeling. On this basis, a self-attention semantic fusion mechanism is designed to integrate the original textual information with image-derived semantics, thereby strengthening the representation capability of textual features. To alleviate completion bias in high missing-rate scenarios, an adversarial generative network that combines a cross-modal generator with a discriminator is developed, where neighbor completion learning and adversarial mechanisms are jointly optimized to simultaneously improve the authenticity of generated features and enhance the robustness of modality completion. Furthermore, a multi-level semantic dynamic fusion strategy is proposed, wherein a hierarchical semantic representation is constructed with gating mechanisms and residual connections to capture both global semantic information and local fine-grained semantics, thereby achieving dynamic cross-level semantic fusion and enhancing the global representation capability of hash codes. Extensive experiments conducted on the MIR Flickr and MS COCO datasets, under both complete-modality and partial-modality scenarios, demonstrate the superiority of the proposed method. Compared with eleven state-of-the-art baselines, including NCH and PMH-F<sup>3</sup>, the proposed approach achieves MAP improvements of 3.1% and 8.9% under complete-modality scenarios, and maximum gains of 7.1% and 16.5% under partial-modality scenarios, thus verifying its effectiveness and robustness.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Partial multimodal hashing with multi-level semantics and adversarial learning

  • 期刊论文

  • 发表日期：

    2026年4月13日

查看全文
  • Hairong Wang,
  • Zhenye Yang,
  • Zhongxuan Li,
  • Jing Wang,
  • Hongying Li

摘要

To address the limitations observed in existing partial multimodal hashing methods, including the insufficient representation capability of multimodal features and hash codes as well as the substantial bias in modality completion under high missing rates, this paper proposes a novel approach, termed Partial Multimodal Hashing with Multi-level Semantics and Adversarial Learning. The proposed framework first establishes a semantic-enhanced multimodal feature modeling scheme by jointly exploiting the CLIP and BLIP-2 models for feature extraction, where image-derived textual semantics are further generated to enrich textual semantic modeling. On this basis, a self-attention semantic fusion mechanism is designed to integrate the original textual information with image-derived semantics, thereby strengthening the representation capability of textual features. To alleviate completion bias in high missing-rate scenarios, an adversarial generative network that combines a cross-modal generator with a discriminator is developed, where neighbor completion learning and adversarial mechanisms are jointly optimized to simultaneously improve the authenticity of generated features and enhance the robustness of modality completion. Furthermore, a multi-level semantic dynamic fusion strategy is proposed, wherein a hierarchical semantic representation is constructed with gating mechanisms and residual connections to capture both global semantic information and local fine-grained semantics, thereby achieving dynamic cross-level semantic fusion and enhancing the global representation capability of hash codes. Extensive experiments conducted on the MIR Flickr and MS COCO datasets, under both complete-modality and partial-modality scenarios, demonstrate the superiority of the proposed method. Compared with eleven state-of-the-art baselines, including NCH and PMH-F3, the proposed approach achieves MAP improvements of 3.1% and 8.9% under complete-modality scenarios, and maximum gains of 7.1% and 16.5% under partial-modality scenarios, thus verifying its effectiveness and robustness.