Textual information provides valuable spatial and scale-level priors for medical image segmentation. However, most existing methods fail to fully exploit the shared spatial and scale-aware cues between visual and textual modalities. To address this limitation, we propose MCASS-Net, a novel Multimodal Co-Aware Scale-Spatial Network that integrates visual and textual features for precise and context-aware lesion segmentation. Specifically, we design two key modules to enhance multimodal representation. The Multimodal Collaborative Scale Alignment (MCSA) module introduces a text-guided, scale-enhanced path via channel attention to enable precise multi-scale feature refinement. The Multimodal Cross-Spatial Synchronization (MCSS) module progressively aligns spatial information across modalities, ensuring consistent spatial understanding between text and image. In addition, we incorporate a semantic consistency–based similarity loss to strengthen cross-modal feature alignment and sharpen lesion localization. Experiments on multiple public medical image datasets demonstrate that MCASS-Net outperforms existing single-modal and multimodal segmentation methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multimodal Co-aware Scale-Spatial Network for Medical Image Segmentation

  • Enming Huang,
  • Teng Fei Gong,
  • Yaxiong Chen,
  • Shengwu Xiong

摘要

Textual information provides valuable spatial and scale-level priors for medical image segmentation. However, most existing methods fail to fully exploit the shared spatial and scale-aware cues between visual and textual modalities. To address this limitation, we propose MCASS-Net, a novel Multimodal Co-Aware Scale-Spatial Network that integrates visual and textual features for precise and context-aware lesion segmentation. Specifically, we design two key modules to enhance multimodal representation. The Multimodal Collaborative Scale Alignment (MCSA) module introduces a text-guided, scale-enhanced path via channel attention to enable precise multi-scale feature refinement. The Multimodal Cross-Spatial Synchronization (MCSS) module progressively aligns spatial information across modalities, ensuring consistent spatial understanding between text and image. In addition, we incorporate a semantic consistency–based similarity loss to strengthen cross-modal feature alignment and sharpen lesion localization. Experiments on multiple public medical image datasets demonstrate that MCASS-Net outperforms existing single-modal and multimodal segmentation methods.