Vision Mamba (VMamba) has recently attracted great research attention due to its ability to obtain a global receptive field with linear computational complexity. However, similar to Vision Transformer (ViT), due to its mechanism of dividing patches, it also faces the issue of insufficient description ability of local details. To address this issue, we design in this paper a dual-stream network that combines VMamba and CNN, aiming to enable the network to possess both the global receptive field of VMamba and the local detail description capability of CNN. Both of the two characteristics are crucial for remote sensing image semantic segmentation. The two streams are supervised and trained through independent loss functions. On the other hand, to enable sufficient information exchange between the two branches, we introduce an auto-scaling fusion module aiming at bridging the semantic gap between VMamba and CNN. Experiments demonstrate that the method proposed in this paper outperforms state-of-the-art methods on multiple remote sensing semantic segmentation datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual-Stream Network of Vision Mamba and CNN with Auto-Scaling for Remote Sensing Image Segmentation

  • Shitao Song,
  • Ye Liu,
  • Jintao Su

摘要

Vision Mamba (VMamba) has recently attracted great research attention due to its ability to obtain a global receptive field with linear computational complexity. However, similar to Vision Transformer (ViT), due to its mechanism of dividing patches, it also faces the issue of insufficient description ability of local details. To address this issue, we design in this paper a dual-stream network that combines VMamba and CNN, aiming to enable the network to possess both the global receptive field of VMamba and the local detail description capability of CNN. Both of the two characteristics are crucial for remote sensing image semantic segmentation. The two streams are supervised and trained through independent loss functions. On the other hand, to enable sufficient information exchange between the two branches, we introduce an auto-scaling fusion module aiming at bridging the semantic gap between VMamba and CNN. Experiments demonstrate that the method proposed in this paper outperforms state-of-the-art methods on multiple remote sensing semantic segmentation datasets.