LEAFusion: An Infrared and Visible Light Image Fusion Network Resilient to Harsh Light Environment Interference Based on Harsh Light Environment Aware

The integration of imaging results from infrared and visible sensors can offer substantial assistance for nighttime vehicle assistance driving. Infrared and visible fusion images contain information from different modalities, including abundant texture details, clearly emphasizing target objects and significantly improving the semantic information and interpretability of a scene. However, mainstream fusion algorithms predominantly focus on the qualitative visual perception of fused images and the quantitative estimation results, neglecting a network’s resistance to interference in harsh environments and the requirements for high-level vision tasks such as segmentation and detection. In response to these problems and enhance the quality of image fusion in nighttime harsh backlight environments, we designed a harsh light environment-aware fusion network (LEAFusion) that is resistant to harsh light environment interference and driven by high-level visual tasks to explore the feasibility of enhancing the network’s robustness against harsh light interference and improving the image fusion performance under harsh backlight conditions. Specifically, we designed a differential feature fusion module (DFFM) to preserve important differential features that are prone to being overlooked. Moreover, we proposed a harsh light environment-aware (HLEA) module to avoid severe degradation of the fusion image quality under harsh nighttime lighting environments. Finally, we designed a cooperative training strategy for the fusion network driven by segmentation tasks presented in this paper. Extensive experiments demonstrate that, compared to other advanced fusion algorithms, our LEAFusion algorithm exhibits significant advantages in fusion in harsh nighttime light conditions, and the results can serve as better inputs for intelligent driving assistance tasks.