Infrared and visible image fusion aims to generate a composite image that simultaneously contains thermal radiation information from infrared images and fine texture details from visible images, thereby enabling high-resolution scene perception, target segmentation, and accurate detection under adverse weather and other complex environmental conditions. However, existing fusion methods generally struggle to simultaneously model local self-similarity and long-range dependencies within images, resulting in insufficient complementarity of features across multiple layers or modules, low information transmission efficiency, and ultimately, limited detail representation and suboptimal fusion performance. To address this issue, this paper proposes a novel infrared and visible image fusion algorithm, named GDTFusion, which integrates CNN with a Transformer architecture. The method is designed to fully exploit and fuse local structural features and global semantic information from the source images, enabling efficient cross-modal information interaction. First, an Interactive Gated Hybrid Attention (IGHA) module based on CNN is designed to extract high-quality visual features, effectively enhancing the texture representation, contrast, and color perception of the fused image. Second, a Transformer-based Multi-Scale Interactive Attention (TMSIA) module is designed to capture local details at different scales and long-range global dependencies, and then fuse them effectively, thereby enhancing the representation capability and integration efficiency of cross-modal features. Extensive comparative experiments were conducted on two public datasets, FMB and MFNet, and the results demonstrate that the proposed GDTFusion method is both effective and superior compared to several state-of-the-art infrared and visible image fusion approaches.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

GDTFusion: Gated Dual-Branch Attention Transformer Network for Infrared and Visible Image Fusion

  • Xuedong Guo,
  • Maoyong Li,
  • Yingying Gao,
  • Lei Deng,
  • Mingli Dong,
  • Lianqing Zhu

摘要

Infrared and visible image fusion aims to generate a composite image that simultaneously contains thermal radiation information from infrared images and fine texture details from visible images, thereby enabling high-resolution scene perception, target segmentation, and accurate detection under adverse weather and other complex environmental conditions. However, existing fusion methods generally struggle to simultaneously model local self-similarity and long-range dependencies within images, resulting in insufficient complementarity of features across multiple layers or modules, low information transmission efficiency, and ultimately, limited detail representation and suboptimal fusion performance. To address this issue, this paper proposes a novel infrared and visible image fusion algorithm, named GDTFusion, which integrates CNN with a Transformer architecture. The method is designed to fully exploit and fuse local structural features and global semantic information from the source images, enabling efficient cross-modal information interaction. First, an Interactive Gated Hybrid Attention (IGHA) module based on CNN is designed to extract high-quality visual features, effectively enhancing the texture representation, contrast, and color perception of the fused image. Second, a Transformer-based Multi-Scale Interactive Attention (TMSIA) module is designed to capture local details at different scales and long-range global dependencies, and then fuse them effectively, thereby enhancing the representation capability and integration efficiency of cross-modal features. Extensive comparative experiments were conducted on two public datasets, FMB and MFNet, and the results demonstrate that the proposed GDTFusion method is both effective and superior compared to several state-of-the-art infrared and visible image fusion approaches.