An Automatic 3D Semantic Occupancy Generation Pipeline for Autonomous Driving
摘要
3D Scene understanding plays a crucial role in autonomous driving. In recent years, the 3D semantic occupancy grid has emerged as a promising scene representation, superseding traditional bounding boxes and garnering substantial research attention. However, generating accurate 3D semantic occupancy annotations necessitates extensive manual labeling and relies on annotated sources such as semantic LiDAR points and 3D instance bounding boxes with unique IDs. The annotating process is notably time-consuming and costly, particularly when dealing with sparse point cloud data. To mitigate this annotation issue, this paper proposes an automatic 3D semantic occupancy generation pipeline that integrates a Vision Foundation Model (VFM) and a point cloud-based 3D object tracker. Our pipeline first leverages the image-based VFM to assign semantic labels to static LiDAR points. Subsequently, a 3D tracker provides unique-ID bounding boxes for movable objects. The final 3D semantic occupancy grid is then constructed by voxelizing the aggregated multi-frame static scene points and dynamic object points. This entire process operates fully automatically, eliminating the need for any additional manual intervention. Comprehensive experiments on the nuScenes dataset demonstrate the effectiveness of our proposed pipeline.