DLPFusion:Natural Language Prompted Infrared and Visible Image Fusion in Drone Scenarios
摘要
Drones, leveraging advantages such as high maneuverability, operational flexibility, and portability, have emerged as the optimal solution for surveillance of critical areas. However, existing multi-modal image fusion methods in drone applications predominantly focus on extracting complementary information while neglecting the detrimental impact of redundant information. Furthermore, due to the absence of authentic fused images, conventional mathematically-defined loss functions fail to accurately model fused image characteristics. To address these limitations, this paper proposes a natural language prompted infrared and visible image fusion network in drone scenarios. First, a dual feature enhancement module (SEM-LEM) extracts high-quality multi-level features. Subsequently, a Gated Attention Mutual Adaptation module (GAMA) and a Multi-scale Cross-Attention Fusion Module (MCAFM) comprehensively integrate multi-scale features. Finally, we propose a language-driven loss function using CLIP(Contrastive Language-Image Pre-training) to bridge modalities, dynamically steering fusion with linguistic descriptors. Experimental results demonstrate that DLPFusion preserves superior textural details and richer semantic information compared to other state-of-the-art methods on both MSRS and DroneVehicle datasets, achieving an SSIM of 0.954 and MI of 4.077 on the MSRS dataset.