Scholarlink

<p>Current multimodal image fusion methods typically cannot precisely locate and fuse features of key regions, leading to artifacts in fusion results and relatively weak local feature expression, failing to ensure the integrity of image structures or target contours. To address this issue, we propose a universal fusion framework called AdverFuse. Based on the Mamba module, this framework introduces an adaptive weight mixed attention mechanism module specifically targeting fusion artifact problems. The module enhances cross-modal feature complementarity through channel branches, precisely locates target regions using spatial attention, and adjusts modal contributions based on the confidence scores output by spatial and channel attention mechanisms, making feature enhancement more aligned with the characteristics of different modal data. Additionally, we design an adversarial network feature enhancement registration module to improve the local feature expression capability of fused images. Combined with the adversarial training mechanism of the discriminator, this module adaptively balances the contribution weights of modalities such as infrared and visible light, maps features of different modalities to a unified semantic space, and extracts richer semantic features while preserving local detail information. We design a convolutional attention module to achieve more comprehensive feature interaction and reduce computational complexity. Experimental results across multiple datasets demonstrate that this method has significant advantages and outperforms SOTA methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AdverFuse: robust fusion of multimodal images based on dynamic attention and adversarial learning

  • 期刊论文

  • 发表日期：

    2026年4月8日

查看全文
  • Fangyan Zhang,
  • Fan Zhang,
  • Yingbing Liu,
  • Fei Ma,
  • Chunsheng Hu

摘要

Current multimodal image fusion methods typically cannot precisely locate and fuse features of key regions, leading to artifacts in fusion results and relatively weak local feature expression, failing to ensure the integrity of image structures or target contours. To address this issue, we propose a universal fusion framework called AdverFuse. Based on the Mamba module, this framework introduces an adaptive weight mixed attention mechanism module specifically targeting fusion artifact problems. The module enhances cross-modal feature complementarity through channel branches, precisely locates target regions using spatial attention, and adjusts modal contributions based on the confidence scores output by spatial and channel attention mechanisms, making feature enhancement more aligned with the characteristics of different modal data. Additionally, we design an adversarial network feature enhancement registration module to improve the local feature expression capability of fused images. Combined with the adversarial training mechanism of the discriminator, this module adaptively balances the contribution weights of modalities such as infrared and visible light, maps features of different modalities to a unified semantic space, and extracts richer semantic features while preserving local detail information. We design a convolutional attention module to achieve more comprehensive feature interaction and reduce computational complexity. Experimental results across multiple datasets demonstrate that this method has significant advantages and outperforms SOTA methods.