Scholarlink

<p>Inspired by the success of the Transformer in NLP and 2D vision, Transformer-based architecture networks have recently become the focus of research in the 3D vision community. However, directly applying Transformer to 3D tasks, especially 3D scene segmentation, will result in huge computational costs due to quadratic computation complexity to the number of input points. Therefore, most existing Transformer-based networks focus on aggregating local features but fail to capture long-range dependencies due to the expensive calculation of global attention. In this paper, we propose a Hierarchical Multi-Token Transformer (HMTT) for point cloud scene segmentation that introduces multiple learnable token at different level to capture long-range dependencies with a low computational cost. Specifically, we first propose a novel Transformer layer, the LFGS-Attention module, which consists of the Local Feature Attention&#xa0;(LFA) module and the Global Semantic Attention&#xa0;(GSA) module to learn local neighborhood features and global semantic contextual information, respectively. Instead of computing global self-attention, the GSA module introduces multiple-category learnable tokens and calculates cross-attention between these tokens and all the points, which reduces memory complexity from <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(\mathcal {O}({N^2})\)</EquationSource> </InlineEquation> to <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(\mathcal {O}(N)\)</EquationSource> </InlineEquation>. The global category features of these category tokens are assigned to corresponding points, and are updated by aggregating the point features at the same time. Based on the proposed LFGS-Attention module, we build a hierarchical architecture transformer to achieve multi-scale feature learning. Extensive experiments demonstrate that the proposed HMTT achieves superior segmentation performance on S3DIS and ScanNetV2 datasets, and the memory consumption of multi-token for capturing global information reduced by 81.6% compared to standard global attention with the same number of input points.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

A hierarchical multi-token transformer for 3D scene segmentation

  • 期刊论文

  • 发表日期：

    2026年2月16日

查看全文
  • Hui Lin,
  • Huibo Li,
  • Ying Zhang,
  • Xuejiao Wang,
  • Yangli Wei,
  • Congcong Wen

摘要

Inspired by the success of the Transformer in NLP and 2D vision, Transformer-based architecture networks have recently become the focus of research in the 3D vision community. However, directly applying Transformer to 3D tasks, especially 3D scene segmentation, will result in huge computational costs due to quadratic computation complexity to the number of input points. Therefore, most existing Transformer-based networks focus on aggregating local features but fail to capture long-range dependencies due to the expensive calculation of global attention. In this paper, we propose a Hierarchical Multi-Token Transformer (HMTT) for point cloud scene segmentation that introduces multiple learnable token at different level to capture long-range dependencies with a low computational cost. Specifically, we first propose a novel Transformer layer, the LFGS-Attention module, which consists of the Local Feature Attention (LFA) module and the Global Semantic Attention (GSA) module to learn local neighborhood features and global semantic contextual information, respectively. Instead of computing global self-attention, the GSA module introduces multiple-category learnable tokens and calculates cross-attention between these tokens and all the points, which reduces memory complexity from \(\mathcal {O}({N^2})\) to \(\mathcal {O}(N)\) . The global category features of these category tokens are assigned to corresponding points, and are updated by aggregating the point features at the same time. Based on the proposed LFGS-Attention module, we build a hierarchical architecture transformer to achieve multi-scale feature learning. Extensive experiments demonstrate that the proposed HMTT achieves superior segmentation performance on S3DIS and ScanNetV2 datasets, and the memory consumption of multi-token for capturing global information reduced by 81.6% compared to standard global attention with the same number of input points.