Kds-radfnet: A distributed thermal infrared and visible image fusion framework based on knowledge distillation and semantic segmentation
摘要
Conventional deep neural network-based methods for fusing thermal infrared and visible images often incur knowledge loss, negatively impacting fused image quality. To address this limitation, this paper presents a novel method named KDS-RADFNet, employing knowledge distillation to fuse these modalities. The approach features a specifically designed knowledge distillation network architecture. A teacher network, utilizing a dual-branch structure, separately extracts thermal radiation features from thermal infrared images and gradient texture features from visible images. Complementary knowledge, formed after cross-modal feature alignment, serves as supervisory signals. A student network learns and integrates this information. A distillation loss function facilitates the transfer of the teacher’s feature representations to the student network, establishing a balance between thermal radiation sensitivity and textural-structural fidelity. This process effectively suppresses feature interference during the fusion of multimodal source images. Furthermore, an unsupervised semantic segmentation network is cascaded with the fusion network. Joint supervision using semantic loss and fusion loss guides the training, enhancing the utility of the fused images for downstream high-level vision tasks. Extensive experiments on multiple public datasets demonstrate that the fusion network enhanced by the knowledge distillation method encodes knowledge more effectively, leading to improved fusion performance.