Scholarlink

Vision-based 3D occupancy can predict a rich representation of the environment around an autonomous vehicle without relying on predefined object classes. An important strategy for this is leveraging the Bird's Eye View to formulate a model of voxel occupancy. However, existing networks for cameras-only in BEV space rely on supervision and need annotation for all scenes, a time-consuming process. To address this issue, we proposed a two-stage learning strategy based on the combination of self-supervision and full-supervision. First, in the self-supervised pre-training phase, high-level 2D image features are extracted using the DINOv2 self-supervised image base model, and these features are projected into the 3D voxel space to enhance the semantic understanding of the model. Then, in the supervised training phase, the pre-trained model are applied for fine-tuning to further learn the feature representations predicted by the occupancy network and to finely determine the occupancy of each voxel. In addition, we also proposed an occupancy network called SGLFT-Occ based on a hybrid CNN and Global Local Flatten Transformer architecture, which accelerates the ability of self-attention mechanism, and this approach significantly improves the extraction efficiency of 3D multi-scale features. Experimental results demonstrate that SGLFT-Occ outperforms existing state-of-the-art methods on the 3D-nuScenes and Semantic KITTI datasets.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

SGLFT-Occ: 3D Occupancy Prediction with Self-supervised Global Local Flatten Transformer

  • 会议论文

  • 发表日期：

    2025年

查看全文
  • Jianping Li,
  • Xiang Wang,
  • Yanli Wang,
  • Guozhen Tan

摘要

Vision-based 3D occupancy can predict a rich representation of the environment around an autonomous vehicle without relying on predefined object classes. An important strategy for this is leveraging the Bird's Eye View to formulate a model of voxel occupancy. However, existing networks for cameras-only in BEV space rely on supervision and need annotation for all scenes, a time-consuming process. To address this issue, we proposed a two-stage learning strategy based on the combination of self-supervision and full-supervision. First, in the self-supervised pre-training phase, high-level 2D image features are extracted using the DINOv2 self-supervised image base model, and these features are projected into the 3D voxel space to enhance the semantic understanding of the model. Then, in the supervised training phase, the pre-trained model are applied for fine-tuning to further learn the feature representations predicted by the occupancy network and to finely determine the occupancy of each voxel. In addition, we also proposed an occupancy network called SGLFT-Occ based on a hybrid CNN and Global Local Flatten Transformer architecture, which accelerates the ability of self-attention mechanism, and this approach significantly improves the extraction efficiency of 3D multi-scale features. Experimental results demonstrate that SGLFT-Occ outperforms existing state-of-the-art methods on the 3D-nuScenes and Semantic KITTI datasets.