The modality missing problem leads to incomplete input data in multi-modal MRI images, severely affecting segmentation performance. Although some methods attempt to tackle this issue, they fail to effectively capture the differences and complementarity between different inputs, resulting in poor performance in scenarios with various modality missing conditions. To address this challenge, we propose a new framework based on a multi-expert collaborative model, which fully integrates the complementary advantages of different inputs, aiming to maximize the use of available information and minimize the negative impact of missing modalities. Specifically, during the training process, we designed independent encoder and decoder modules for each modality to focus on extracting features from individual modalities while ensuring regularization. Next, we constructed a multi-expert collaborative model, consisting of both multi-modal and single-modal expert groups. The multi-modal expert group selects the most suitable expert to handle tasks based on the missing modality conditions, and learns the correlations between different modalities through expert collaboration. Meanwhile, the single-modal expert group focuses on processing single-modal features. Finally, we combine the complementary information from both single-modal and multi-modal features and input them into the decoder for processing. Experimental results on the BraTS2018 and BraTS2020 datasets indicate that the framework significantly improves brain tumor segmentation performance under modality missing conditions, demonstrating superior robustness and accuracy compared to existing methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Incomplete Multi-modal Brain Tumor Segmentation via Multi-expert Collaboration

  • Zhengyi Liu,
  • Rui Zhang,
  • Jinhai Yu,
  • Xianyong Fang,
  • Linbo Wang

摘要

The modality missing problem leads to incomplete input data in multi-modal MRI images, severely affecting segmentation performance. Although some methods attempt to tackle this issue, they fail to effectively capture the differences and complementarity between different inputs, resulting in poor performance in scenarios with various modality missing conditions. To address this challenge, we propose a new framework based on a multi-expert collaborative model, which fully integrates the complementary advantages of different inputs, aiming to maximize the use of available information and minimize the negative impact of missing modalities. Specifically, during the training process, we designed independent encoder and decoder modules for each modality to focus on extracting features from individual modalities while ensuring regularization. Next, we constructed a multi-expert collaborative model, consisting of both multi-modal and single-modal expert groups. The multi-modal expert group selects the most suitable expert to handle tasks based on the missing modality conditions, and learns the correlations between different modalities through expert collaboration. Meanwhile, the single-modal expert group focuses on processing single-modal features. Finally, we combine the complementary information from both single-modal and multi-modal features and input them into the decoder for processing. Experimental results on the BraTS2018 and BraTS2020 datasets indicate that the framework significantly improves brain tumor segmentation performance under modality missing conditions, demonstrating superior robustness and accuracy compared to existing methods.