Point cloud analysis is a fundamental task in computer vision. Transformer-based methods are increasingly popular in point cloud analysis due to their strong global context modeling ability, but they have limitations in capturing local geometric features. Recently, Mamba-based methods have emerged with comparable performance and better efficiency, yet they fall short in modeling global context like Transformer. In addition, Mamba’s ability to capture local geometric features has not been fully exploited. How to fully exploit Mamba’s geometric modeling capability while integrating the strengths of both architectures is a point worth exploring. In this paper, we propose a hybrid point cloud analysis framework combining Transformer and Mamba to jointly learn local geometric features and global contextual features. The unordered nature of point clouds prevents Mamba from effectively extracting local geometric features through sequential modeling. To address this, we introduce a spatial proximity-based reordering strategy to arrange point cloud patches into a meaningful sequence. Based on this, we design a Mamba-Transformer Mixer with alternately stacked Mamba and Transformer layers to achieve enhanced feature extraction. Experimental results show that our method outperforms both Transformer-based and Mamba-based models across multiple datasets, including ScanObjectNN, ModelNet40, and ShapeNetPart, demonstrating the effectiveness of the Mamba-Transformer hybrid architecture. The code will be released soon.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

PointMM: A Hybrid Mamba-Transformer Framework for Point Cloud Analysis with Morton Reordering Strategy

  • Jianhui Li,
  • Changwei Wang,
  • Shujun Gu,
  • Zhiwei Yang,
  • Chuanfu Wu,
  • Longxiang Gao,
  • Kexue Fu,
  • Youyang Qu

摘要

Point cloud analysis is a fundamental task in computer vision. Transformer-based methods are increasingly popular in point cloud analysis due to their strong global context modeling ability, but they have limitations in capturing local geometric features. Recently, Mamba-based methods have emerged with comparable performance and better efficiency, yet they fall short in modeling global context like Transformer. In addition, Mamba’s ability to capture local geometric features has not been fully exploited. How to fully exploit Mamba’s geometric modeling capability while integrating the strengths of both architectures is a point worth exploring. In this paper, we propose a hybrid point cloud analysis framework combining Transformer and Mamba to jointly learn local geometric features and global contextual features. The unordered nature of point clouds prevents Mamba from effectively extracting local geometric features through sequential modeling. To address this, we introduce a spatial proximity-based reordering strategy to arrange point cloud patches into a meaningful sequence. Based on this, we design a Mamba-Transformer Mixer with alternately stacked Mamba and Transformer layers to achieve enhanced feature extraction. Experimental results show that our method outperforms both Transformer-based and Mamba-based models across multiple datasets, including ScanObjectNN, ModelNet40, and ShapeNetPart, demonstrating the effectiveness of the Mamba-Transformer hybrid architecture. The code will be released soon.