Scholarlink

<p>In this paper, a Hierarchical Cross-Modal Alignment Optimization Framework (HCAOF) is proposed with the intention of enhancing the robustness and consistency of semantic relationships in multimodal translation. The model incorporates a Bidirectional Adversarial Alignment Mechanism (BAAM) for fine-grained visual-textual alignment, a Dynamic Semantic Enhancement Network (DSEN) for noise filtering and spatial modeling, and a reprojection gradient optimization for managing long-tail and weakly aligned samples. All of these components work together to achieve the desired results. The approach was evaluated using the MS-COCO, Flickr30K, and TextILE datasets, and it achieved a BLEU-4 score of 38.3 and a METEOR score of 33.2. This marked a 2.4% and 2.5% increase in comparison to the baseline that was established by the ALBEF. The retrieval accuracy (R@1) reached 58.9% when comparing text to images and 76.3% when comparing images to texts. Simultaneously, the Visual Hallucination Rate decreased to 9.7%. The system is successful for applications that are used in the real world, such as Cross-Modal Retrieval (CMR) and multilingual picture captioning, even when the conditions are noisy or involve a limited number of resources.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Cross-modal Alignment Algorithm and Dynamic Semantic Enhancement in Multimodal Translation

  • 期刊论文

  • 发表日期：

    2026年5月28日

查看全文
  • Qian Wang

摘要

In this paper, a Hierarchical Cross-Modal Alignment Optimization Framework (HCAOF) is proposed with the intention of enhancing the robustness and consistency of semantic relationships in multimodal translation. The model incorporates a Bidirectional Adversarial Alignment Mechanism (BAAM) for fine-grained visual-textual alignment, a Dynamic Semantic Enhancement Network (DSEN) for noise filtering and spatial modeling, and a reprojection gradient optimization for managing long-tail and weakly aligned samples. All of these components work together to achieve the desired results. The approach was evaluated using the MS-COCO, Flickr30K, and TextILE datasets, and it achieved a BLEU-4 score of 38.3 and a METEOR score of 33.2. This marked a 2.4% and 2.5% increase in comparison to the baseline that was established by the ALBEF. The retrieval accuracy (R@1) reached 58.9% when comparing text to images and 76.3% when comparing images to texts. Simultaneously, the Visual Hallucination Rate decreased to 9.7%. The system is successful for applications that are used in the real world, such as Cross-Modal Retrieval (CMR) and multilingual picture captioning, even when the conditions are noisy or involve a limited number of resources.