A Mamba-optimized decoder for improved medical image segmentation
摘要
Both Transformer-based models and the emerging Mamba-based models exhibit a critical limitation: they predominantly focus on optimizing the encoder while largely overlooking the significance of the decoder. This imbalance in encoder-decoder architectures leads to severe structural asymmetry, which in turn constrains overall model performance. We conducted an in-depth study on the structural composition and functional mechanisms of decoders and proposed SegEO Mamba (i.e., SegMamba Enhanced and Optimized by Mamba) to overcome this issue. This model incorporates a learnable interpolation method that integrates encoder features for more effective upsampling, termed Position Offset with Encoder-Guided Adaptive Method for Upsampling (PE-Upsampling). Furthermore, we introduce Bidirectional Mamba Space and Channel Parallel Attention (BMSC Attention) within skip connections to enhance the fusion of encoder and decoder features. By progressively enriching the decoder, our approach restores balance within the encoder-decoder architecture, effectively alleviating performance constraints caused by structural imbalance. Our proposed SegEO Mamba framework demonstrates state-of-the-art performance across multiple medical imaging benchmarks: achieving a 90.97% Dice score for brain tumor segmentation on BraTS2023, maintaining competitive performance on AIIB2023, and notably attaining top-ranking segmentation performance for multiple abdominal organs on the BTCV benchmark. The model’s exceptional multi-organ segmentation capability robustly validates its strong generalization across diverse clinical scenarios. The code and the best model can be accessed at https://github.com/ZeKey780/SegEO.