Joint image registration and fusion aim to align source images and integrate complementary information into a unified output. However, existing approaches often rely on fixed receptive fields and neglect multimodal enhancement and long-range dependencies, resulting in task inconsistency and edge artifacts. To address these challenges, we propose UMIMamba, a unified Mamba-based framework for unaligned multimodal image registration and fusion. Specifically, to correct spatial distortion, the Progressive Refinement Registration Module (PRRM) estimates deformation fields via multi-scale optical flow registration. We design fusion network based on Mamba, the Deep Multimodal Fusion (DMF) module captures global context and edge details to enhance complementary features while preserving textures. Additionally, we design Separable Deformable Convolution (SDConv) with group-wise learnable offsets to adaptively adjust receptive fields for better feature representation. Extensive experiments on unaligned multimodal datasets show that UMIMamba consistently outperforms state-of-the-art methods in both registration accuracy and fusion quality.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

UMIMamba: Differential Feature Enhancement with Mamba for Unaligned Multimodal Image Registration and Fusion

  • Yunde Zhang,
  • Jun Kong

摘要

Joint image registration and fusion aim to align source images and integrate complementary information into a unified output. However, existing approaches often rely on fixed receptive fields and neglect multimodal enhancement and long-range dependencies, resulting in task inconsistency and edge artifacts. To address these challenges, we propose UMIMamba, a unified Mamba-based framework for unaligned multimodal image registration and fusion. Specifically, to correct spatial distortion, the Progressive Refinement Registration Module (PRRM) estimates deformation fields via multi-scale optical flow registration. We design fusion network based on Mamba, the Deep Multimodal Fusion (DMF) module captures global context and edge details to enhance complementary features while preserving textures. Additionally, we design Separable Deformable Convolution (SDConv) with group-wise learnable offsets to adaptively adjust receptive fields for better feature representation. Extensive experiments on unaligned multimodal datasets show that UMIMamba consistently outperforms state-of-the-art methods in both registration accuracy and fusion quality.