Saliency guided progressive fusion of infrared and polarization for military images with complex backgrounds\(^{\star }\)

Thermal targets stand out on infrared light images but infrared imaging is vulnerable to thermal noise, thus the edges of targets in infrared images are often blurred. On the other hand, visible polarized images could provide rich background texture features and enhance the edge details of targets. In this paper, we propose a saliency-guided end-to-end fusion framework called SGfusionNet for fusion of infrared and visible polarized images, which aims to preserve more comprehensive and informative fusion images. Firstly, we introduce the feature extraction module of the dual branches, which embed gradient filters to better capture strong features. Secondly, we introduce an adaptive feature fusion module to achieve complementary fusion by learning to select the dominant features from multi-modal images progressively. Finally, we design a saliency-weighted loss that assigns higher weights to semantically salient regions, thus guiding feature extraction and fusion to focus more on salient regions while retaining rich information in the non-salient regions. Substantial experiments have demonstrated that our fusion method achieves high-quality fusion results with more natural/superior visual effects. Compared to the recently well-performing STDfusion algorithm, our framework exhibits an average increase of 12.28%, 18.72%, 31.03%, and 10.23% in metrics of mutual information (MI), average gradient (AG), spatial frequency (SF), and standard deviation (SD), respectively. The source code will be available at https://github.com/Lao/SGfusionNet.