Infrared and visible image fusion based on Haar wavelet downsampling and multi-scale feature aggregation
摘要
Infrared imaging devices have shortcomings in obtaining detailed information, while visible imaging devices are highly susceptible to environmental factors like weather changes and lighting conditions. The image fusion process optimally combines infrared thermal data with visible-light texture information to produce an enriched composite image containing complementary multimodal features. However, the current CNN-based fusion approaches usually encounter issues related to feature information loss and feature fusion limitations. To solve these problems, we propose a multi-scale information aggregation network (MIANet) to effectively extract features and use multi-scale feature information for infrared and visible image fusion. Specifically, the MIANet is essentially composed of three components, namely, lossless feature extractor (LFE) , multi-scale feature aggregation (MFA) module and image reconstructor. The LFE primarily incorporates Haar wavelet downsampling (HWD) modules, which designed to maximize information retention during spatial resolution reduction of feature representations. In addition, we introduce the MFA module to integrate features at different scales and capture the global context information of images. Experimental results on various datasets demonstrate that the MIANet not only surpasses existing advanced methods in quantitative evaluation of indicators, but also exhibits better visual effects in qualitative analysis.