Multimodal classification in medical imaging faces two key challenges in cross-modal feature fusion: significant semantic gaps between modalities often cause loss of critical pathological information during fusion, and existing diagnostic models generally process cases individually without leveraging population-level diagnostic knowledge. To address these issues, we propose the Dual-Feature Interaction Graph Neural Network (DFI-GNN). Our approach integrates feature interaction fusion, collective knowledge mining, and collaborative optimization. Specifically, we first introduce a dual-feature interaction fusion module that preserves original discriminative features through differentiable regularization constraints and residual feature backpropagation. Second, we construct patient association graphs based on multimodal feature similarity to mine population-level diagnostic knowledge via graph attention networks. Finally, a multimodal-graph collaborative learning strategy jointly optimizes multimodal loss, graph relational loss, and fusion regularization loss, enhancing single-modal discriminability, collective knowledge relevance, and cross-modal fusion robustness. Extensive experiments on multiple benchmarks show that DFI-GNN achieves superior performance in multimodal feature learning, offering a more reliable solution for medical image diagnosis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DFI-GNN: A Dual-Feature Interaction Graph Neural Network Model for Multimodal Medical Image Classification

  • Meng Yu,
  • Lijuan Sun,
  • Yutong Gao,
  • Gaohu Li,
  • Jingchen Wu,
  • Xu Wu

摘要

Multimodal classification in medical imaging faces two key challenges in cross-modal feature fusion: significant semantic gaps between modalities often cause loss of critical pathological information during fusion, and existing diagnostic models generally process cases individually without leveraging population-level diagnostic knowledge. To address these issues, we propose the Dual-Feature Interaction Graph Neural Network (DFI-GNN). Our approach integrates feature interaction fusion, collective knowledge mining, and collaborative optimization. Specifically, we first introduce a dual-feature interaction fusion module that preserves original discriminative features through differentiable regularization constraints and residual feature backpropagation. Second, we construct patient association graphs based on multimodal feature similarity to mine population-level diagnostic knowledge via graph attention networks. Finally, a multimodal-graph collaborative learning strategy jointly optimizes multimodal loss, graph relational loss, and fusion regularization loss, enhancing single-modal discriminability, collective knowledge relevance, and cross-modal fusion robustness. Extensive experiments on multiple benchmarks show that DFI-GNN achieves superior performance in multimodal feature learning, offering a more reliable solution for medical image diagnosis.