Scholarlink

<p>The extraction and fusion of multi-scale information represents a crucial aspect of semantic segmentation. With the significant advancements in computational capabilities, using Convolutional Neural Networks (CNNs) for extracting multi-scale contextual information has become a popular approach in this field.In recent years, many Visual Transformer (ViT) models have emerged, largely inspired by the adaptation of Transformer architectures originally developed for Natural Language Processing (NLP) to image processing tasks. While these models demonstrate impressive performance, they often come with considerable computational costs. In this study, we extract and fuse multi-scale information to enhance model performance while maintaining an acceptable computational expense. We propose the Mix-Dilated Neighborhood Attention (Mix-DINA), which incorporates a token-mixing mechanism. This addition allows the attention model to more effectively capture positional information. We have built the Three-Stream Mix-Dilated Neighborhood Attention (Three-Stream Mix-DINA) module which incorporates the Mix-DINA module. This module captures contextual information at multiple scales using windows of varying sizes. Furthermore, to tackle the issue of long-range feature dependencies, we draw inspiration from Atrous Spatial Pyramid Pooling (ASPP). By implementing neighborhood attention with different dilation rates,we are able to establish long-range dependencies among features. Finally,we propose a generic semantic segmentation decoder named Mix-Dilated Neighborhood Attention Transformer (Mix-DINAT), which can be combined with any existing Hierarchical Vision Transformer (HVT) to create a complete semantic segmentation ViT model. The experimental results demonstrate that Mix-DINAT outperforms existing vision transformer methods in terms of efficiency. Additionally, Mix-DINAT achieves impressive performance on the ADE20K (52.6% mIoU), Cityscapes (83.2% mIoU), and COCO-Stuff (46.8% mIoU) datasets.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Mix-DINAT: Three-Stream transformer design for semantic segmentation with multi-scale representations

  • 期刊论文

  • 发表日期：

    2026年3月24日

查看全文
  • Shuren Zhou,
  • Hao Wu,
  • Yujian Zhu

摘要

The extraction and fusion of multi-scale information represents a crucial aspect of semantic segmentation. With the significant advancements in computational capabilities, using Convolutional Neural Networks (CNNs) for extracting multi-scale contextual information has become a popular approach in this field.In recent years, many Visual Transformer (ViT) models have emerged, largely inspired by the adaptation of Transformer architectures originally developed for Natural Language Processing (NLP) to image processing tasks. While these models demonstrate impressive performance, they often come with considerable computational costs. In this study, we extract and fuse multi-scale information to enhance model performance while maintaining an acceptable computational expense. We propose the Mix-Dilated Neighborhood Attention (Mix-DINA), which incorporates a token-mixing mechanism. This addition allows the attention model to more effectively capture positional information. We have built the Three-Stream Mix-Dilated Neighborhood Attention (Three-Stream Mix-DINA) module which incorporates the Mix-DINA module. This module captures contextual information at multiple scales using windows of varying sizes. Furthermore, to tackle the issue of long-range feature dependencies, we draw inspiration from Atrous Spatial Pyramid Pooling (ASPP). By implementing neighborhood attention with different dilation rates,we are able to establish long-range dependencies among features. Finally,we propose a generic semantic segmentation decoder named Mix-Dilated Neighborhood Attention Transformer (Mix-DINAT), which can be combined with any existing Hierarchical Vision Transformer (HVT) to create a complete semantic segmentation ViT model. The experimental results demonstrate that Mix-DINAT outperforms existing vision transformer methods in terms of efficiency. Additionally, Mix-DINAT achieves impressive performance on the ADE20K (52.6% mIoU), Cityscapes (83.2% mIoU), and COCO-Stuff (46.8% mIoU) datasets.