Segmentation of brain tumors from MRI is essential for accurate brain tumor diagnosis. However, current multimodal brain tumor segmentation approaches often rely on channel concatenation to integrate multimodal MRI images, which restricts the complete use of complementary information between modalities. Consequently, to overcome this limitation, we propose a brain tumor segmentation network based on modal interaction and multimodal fusion. Initially, inspired by the structural correlations among various MRI modalities, the Modality Interaction Block (MI) utilizes Swin Transformer to combine highly correlated modalities for feature extraction. This approach effectively captures long-range dependencies and global associations within the modality combinations. Furthermore, the Multimodal Fusion module (MFF) based on attention mechanisms is proposed. This module adjusts the contribution of each modality feature according to the channel attention weights, enabling the network to focus more on key features. Additionally, a Multi-Scale Feature Calibration (MFC) module is incorporated to enhance segmentation accuracy, creating channel and spatial dependencies across multiple scales to ensure feature consistency. We evaluate the proposed method on the BraTS2021 dataset for multimodal brain tumor segmentation, aiming at the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) regions. The Dice scores can reach 91.579%, 88.153% and 85.472%, respectively. The Hausdorff Distance (HD) values are 6.676 mm, 6.639 mm, and 5.964 mm, respectively. Experimental results demonstrate that our approach exceeds the performance of other methods utilizing CNNs, Transformers, and multimodal fusion techniques.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

The Multimodality MRI Brain Tumor Segmentation Network Based on Modal Interaction and Multimodal Fusion

  • GuoDong Zhang,
  • ZhiHui Liu,
  • Wei Guo,
  • ZhaoXuan Gong,
  • RongHui Ju

摘要

Segmentation of brain tumors from MRI is essential for accurate brain tumor diagnosis. However, current multimodal brain tumor segmentation approaches often rely on channel concatenation to integrate multimodal MRI images, which restricts the complete use of complementary information between modalities. Consequently, to overcome this limitation, we propose a brain tumor segmentation network based on modal interaction and multimodal fusion. Initially, inspired by the structural correlations among various MRI modalities, the Modality Interaction Block (MI) utilizes Swin Transformer to combine highly correlated modalities for feature extraction. This approach effectively captures long-range dependencies and global associations within the modality combinations. Furthermore, the Multimodal Fusion module (MFF) based on attention mechanisms is proposed. This module adjusts the contribution of each modality feature according to the channel attention weights, enabling the network to focus more on key features. Additionally, a Multi-Scale Feature Calibration (MFC) module is incorporated to enhance segmentation accuracy, creating channel and spatial dependencies across multiple scales to ensure feature consistency. We evaluate the proposed method on the BraTS2021 dataset for multimodal brain tumor segmentation, aiming at the whole tumor (WT), tumor core (TC), and enhancing tumor (ET) regions. The Dice scores can reach 91.579%, 88.153% and 85.472%, respectively. The Hausdorff Distance (HD) values are 6.676 mm, 6.639 mm, and 5.964 mm, respectively. Experimental results demonstrate that our approach exceeds the performance of other methods utilizing CNNs, Transformers, and multimodal fusion techniques.