<p>Drones, leveraging advantages such as high maneuverability, operational flexibility, and portability, have emerged as the optimal solution for surveillance of critical areas. However, existing multi-modal image fusion methods in drone applications predominantly focus on extracting complementary information while neglecting the detrimental impact of redundant information. Furthermore, due to the absence of authentic fused images, conventional mathematically-defined loss functions fail to accurately model fused image characteristics. To address these limitations, this paper proposes a natural language prompted infrared and visible image fusion network in drone scenarios. First, a dual feature enhancement module (SEM-LEM) extracts high-quality multi-level features. Subsequently, a Gated Attention Mutual Adaptation module (GAMA) and a Multi-scale Cross-Attention Fusion Module (MCAFM) comprehensively integrate multi-scale features. Finally, we propose a language-driven loss function using CLIP(Contrastive Language-Image Pre-training) to bridge modalities, dynamically steering fusion with linguistic descriptors. Experimental results demonstrate that DLPFusion preserves superior textural details and richer semantic information compared to other state-of-the-art methods on both MSRS and DroneVehicle datasets, achieving an SSIM of 0.954 and MI of 4.077 on the MSRS dataset.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DLPFusion:Natural Language Prompted Infrared and Visible Image Fusion in Drone Scenarios

  • He Xiao,
  • Yingliang Wen,
  • Yaoyi Liu,
  • Jiandong Zhang,
  • Hao Li,
  • Huang Li,
  • Qiuming Liu

摘要

Drones, leveraging advantages such as high maneuverability, operational flexibility, and portability, have emerged as the optimal solution for surveillance of critical areas. However, existing multi-modal image fusion methods in drone applications predominantly focus on extracting complementary information while neglecting the detrimental impact of redundant information. Furthermore, due to the absence of authentic fused images, conventional mathematically-defined loss functions fail to accurately model fused image characteristics. To address these limitations, this paper proposes a natural language prompted infrared and visible image fusion network in drone scenarios. First, a dual feature enhancement module (SEM-LEM) extracts high-quality multi-level features. Subsequently, a Gated Attention Mutual Adaptation module (GAMA) and a Multi-scale Cross-Attention Fusion Module (MCAFM) comprehensively integrate multi-scale features. Finally, we propose a language-driven loss function using CLIP(Contrastive Language-Image Pre-training) to bridge modalities, dynamically steering fusion with linguistic descriptors. Experimental results demonstrate that DLPFusion preserves superior textural details and richer semantic information compared to other state-of-the-art methods on both MSRS and DroneVehicle datasets, achieving an SSIM of 0.954 and MI of 4.077 on the MSRS dataset.