In robotic tasks such as medical surgery and agricultural harvesting, accurate 3D perception of target objects remains challenging due to unstructured environmental occlusions, lighting variations. Pre-reconstructed 3D model of the target can enhance the reliability and accuracy of robotic systems. Existing object-level 3D reconstruction methods often separate scene reconstruction from object detection, relying on manual processing to refine 3D models. We introduce a novel framework for robust 3D object reconstruction from RGB-D sequences. Specifically, we propose a spatiotemporally consistent object segmentation mechanism that integrates a 2D segmentation model with 3D geometric clustering. The PREDATOR model is then leveraged to perform chunk-to-model registration, with a voting-based strategy effectively identifying outlier registration results. Experiments on the TUM dataset demonstrate the framework’s adaptability to challenging scenarios. 3D reconstruction achieves a Chamfer Distance of 0.017 m, and the proposed outlier registration filtering algorithm achieves a precision of 0.96.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Robust Framework for 3D Object Reconstruction via Instance Segmentation and Neural Registration

  • Keming Zhang,
  • Pingguo Cao,
  • Chunyi Xu,
  • Quanjun Song,
  • Yong Chen,
  • Lei Ding

摘要

In robotic tasks such as medical surgery and agricultural harvesting, accurate 3D perception of target objects remains challenging due to unstructured environmental occlusions, lighting variations. Pre-reconstructed 3D model of the target can enhance the reliability and accuracy of robotic systems. Existing object-level 3D reconstruction methods often separate scene reconstruction from object detection, relying on manual processing to refine 3D models. We introduce a novel framework for robust 3D object reconstruction from RGB-D sequences. Specifically, we propose a spatiotemporally consistent object segmentation mechanism that integrates a 2D segmentation model with 3D geometric clustering. The PREDATOR model is then leveraged to perform chunk-to-model registration, with a voting-based strategy effectively identifying outlier registration results. Experiments on the TUM dataset demonstrate the framework’s adaptability to challenging scenarios. 3D reconstruction achieves a Chamfer Distance of 0.017 m, and the proposed outlier registration filtering algorithm achieves a precision of 0.96.