错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

\(\hbox {KD}^{3}\)mt: knowledge distillation-driven dynamic mixer transformer for medical image fusion

  • Zhaijuan Ding,
  • Yanyu Liu,
  • Sen Liu,
  • Kangjian He,
  • Dongming Zhou

摘要

The synergistic combination of multimodal medical images offers a comprehensive representation of biomedical information. However, the main challenge lies in effectively extracting and fusing common and specific features from different images, especially due to the limited availability of labeled data for fusion tasks. In this paper, we propose a novel Knowledge Distillation-Driven Dynamic Mixer Transformer for robust medical image fusion, named \({\textbf {KD}}^{3}\) KD 3 MT. Specifically, the primary theoretical basis of \(\hbox {KD}^{3}\) KD 3 MT is the dynamic ego-context features learning and knowledge distillation architecture. Firstly, we propose a knowledge distillation learning method composed of a pre-trained deep teacher network and a dynamic synergy transformer-based student network. The student network produces effective feature representation and generalization capabilities through the guidance of soft labels generated by the deep teacher network. Subsequently, we design a dynamic mixer transformer (DMT) to mine the local and global correlation clues. Furthermore, we propose a wavelet domain fusion (WDF) module based on multi-scale transformation to accommodate the differences in deep features of medical images. Ultimately, the decoder outputs a promising fused image with abundant tissue details and distinct bone structures. Extensive experimental results demonstrate that the proposed fusion network outperforms the state-of-the-art methods in qualitative and quantitative evaluation. Additionally, our research materials, data, results and code will be accessible for peer reference and free download at https://github.com/YN-yanyul/KD3MT.