Scholarlink

Purpose <p>Natural orifice surgeries minimize the need for incisions and reduce recovery time compared to open surgery; however, they require higher expertise due to visualization and orientation challenges. To enable reliable scene understanding for surgeon guidance and automation, we propose a perception pipeline that generates semantically informed 3D reconstructions.</p> Methods <p>We bring learning-based segmentation, depth estimation, and 3D reconstruction modules together. Through segmentation, we delineate tumor and prostate lobe borders, and through depth estimation and real-time SLAM-based reconstruction, we generate dense 3D point clouds from monocular videos. By propagating 2D labels into 3D space, we create real-time segmented maps of the surgical scenes. Additionally, we use registration with robot poses to solve the scale ambiguity of mapping from monocular images and allow the use of semantically informed real-time reconstructions in robotic surgeries.</p> Results <p>We achieve sub-millimeter reconstruction accuracy based on average one-sided Chamfer distances, average pose registration RMSE of 0.9 mm, and an estimated scale within 2% of ground truth. Compared to offline Structure-from-Motion baseline, the proposed SLAM-based approach improves processing time while maintaining or improving reconstruction accuracy. Qualitative evaluations show robustness in challenging scenarios, including submerged prostate experiments and cadaver airway explorations.</p> Conclusion <p>We present a modular perception pipeline, integrating semantic segmentation with real-time monocular SLAM for natural orifice surgeries. This pipeline offers a promising solution for scene understanding that can facilitate automation or surgeon guidance. Due to its plug-and-play design and demonstrated generalizability across anatomies and experimental conditions, this framework provides a scalable foundation for future clinical translation.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Perseus: perception with semantic endoscopic understanding and SLAM

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月26日

查看全文
  • Ayberk Acar,
  • Fangjie Li,
  • Susheela Sharma Stern,
  • Lidia Al-Zogbi,
  • Hao Li,
  • Kanyifeechukwu Jane Oguine,
  • Dilara Isik,
  • Brendan Burkhart,
  • Jesse F. d’Almeida,
  • Robert J. Webster III,
  • Ipek Oguz,
  • Jie Ying Wu

摘要

Purpose

Natural orifice surgeries minimize the need for incisions and reduce recovery time compared to open surgery; however, they require higher expertise due to visualization and orientation challenges. To enable reliable scene understanding for surgeon guidance and automation, we propose a perception pipeline that generates semantically informed 3D reconstructions.

Methods

We bring learning-based segmentation, depth estimation, and 3D reconstruction modules together. Through segmentation, we delineate tumor and prostate lobe borders, and through depth estimation and real-time SLAM-based reconstruction, we generate dense 3D point clouds from monocular videos. By propagating 2D labels into 3D space, we create real-time segmented maps of the surgical scenes. Additionally, we use registration with robot poses to solve the scale ambiguity of mapping from monocular images and allow the use of semantically informed real-time reconstructions in robotic surgeries.

Results

We achieve sub-millimeter reconstruction accuracy based on average one-sided Chamfer distances, average pose registration RMSE of 0.9 mm, and an estimated scale within 2% of ground truth. Compared to offline Structure-from-Motion baseline, the proposed SLAM-based approach improves processing time while maintaining or improving reconstruction accuracy. Qualitative evaluations show robustness in challenging scenarios, including submerged prostate experiments and cadaver airway explorations.

Conclusion

We present a modular perception pipeline, integrating semantic segmentation with real-time monocular SLAM for natural orifice surgeries. This pipeline offers a promising solution for scene understanding that can facilitate automation or surgeon guidance. Due to its plug-and-play design and demonstrated generalizability across anatomies and experimental conditions, this framework provides a scalable foundation for future clinical translation.