<p>Infrared and visible image fusion aims to generate a fusion image that integrates complementary information from both modalities. Current deep learning-based methods often exhibit a limitation: networks that focus on spatial-domain features usually neglect the detail-enhancing potential of frequency-domain information, whereas purely frequency-domain approaches struggle to maintain semantic coherence with spatial structures. To address this, we propose a novel and efficient fusion model. First, a Swin Transformer-based encoder is employed to extract deep, multi-scale features. Second, a complementary dual-branch “Spatial-Frequency” fusion module is designed. This module comprises a Spatial Attention Fusion Module (SAFM) for refining features in the spatial domain and a novel Wavelet-domain Mamba Fusion Module (WMFM) for processing features in the frequency domain, where the Mamba mechanism synergistically integrates global semantics and local details. Finally, a recurrent neural network architecture is employed in the decoder to preserve the coherence of spatio-temporal information. Quantitative experimental results on three public datasets (TNO, RoadScene, and MSRS) demonstrate that the proposed method achieves top or near-top rankings across multiple evaluation metrics, including <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\({FMI}_{wt}\)</EquationSource> </InlineEquation>, AG, VIF, SSIM, and SCD. These results clearly demonstrate that the proposed fusion method effectively preserves high-frequency details and edge information from source images, yielding fused images with superior visual quality and structural fidelity. This further validates the effectiveness and robustness of the proposed fusion strategy.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

WMambaFuse: an infrared and visible image fusion network based on wavelet mamba

  • Jiashuo Wang,
  • Yu Si,
  • Yong Chen,
  • Xiaoyun Sun,
  • Zhiyu Li,
  • Hui Xing,
  • Mingming Wang

摘要

Infrared and visible image fusion aims to generate a fusion image that integrates complementary information from both modalities. Current deep learning-based methods often exhibit a limitation: networks that focus on spatial-domain features usually neglect the detail-enhancing potential of frequency-domain information, whereas purely frequency-domain approaches struggle to maintain semantic coherence with spatial structures. To address this, we propose a novel and efficient fusion model. First, a Swin Transformer-based encoder is employed to extract deep, multi-scale features. Second, a complementary dual-branch “Spatial-Frequency” fusion module is designed. This module comprises a Spatial Attention Fusion Module (SAFM) for refining features in the spatial domain and a novel Wavelet-domain Mamba Fusion Module (WMFM) for processing features in the frequency domain, where the Mamba mechanism synergistically integrates global semantics and local details. Finally, a recurrent neural network architecture is employed in the decoder to preserve the coherence of spatio-temporal information. Quantitative experimental results on three public datasets (TNO, RoadScene, and MSRS) demonstrate that the proposed method achieves top or near-top rankings across multiple evaluation metrics, including \({FMI}_{wt}\) , AG, VIF, SSIM, and SCD. These results clearly demonstrate that the proposed fusion method effectively preserves high-frequency details and edge information from source images, yielding fused images with superior visual quality and structural fidelity. This further validates the effectiveness and robustness of the proposed fusion strategy.