Scholarlink

Purpose: <p>Markerless navigation in minimally invasive surgery is still an unsolved challenge. Many proposed navigation systems for minimally invasive surgeries rely on stereoscopic images, while in clinical practice oftentimes monocular endoscopes are used. Combined with the lack of automatic video-based navigation systems for prostatectomies, this paper explores methods to tackle both research gaps at the same time for robot-assisted prostatectomies.</p> Methods: <p>In order to realize a semi-automatic augmented reality overlay for navigated prostatectomy, the camera pose w.r.t. the prostate needs to be estimated. We developed a method where visual cues are drawn on top of the organ after an initial manual alignment, simultaneously creating matching landmarks on the 2D and 3D data. Starting from this key frame, the cues are then tracked in the endoscopic video. Both PnPRansac and differentiable rendering are then explored to perform 2D–3D registration for each frame.</p> Results: <p>We performed experiments on synthetic and in vivo data. On synthetic data differentiable rendering can achieve a median target registration error of 6.11 mm. Both PnPRansac and differentiable rendering are feasible methods for 2D–3D registration.</p> Conclusion: <p>We demonstrated a video-based markerless augmented reality overlay for navigated prostatectomy, using visual cues as an anchor.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An augmented reality overlay for navigated prostatectomy using fiducial-free 2D–3D registration

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年5月8日

查看全文
  • Johannes Bender,
  • Jeremy Kwe,
  • Benedikt Hoeh,
  • Katharina Boehm,
  • Ivan Platzek,
  • Angelika Borkowetz,
  • Stefanie Speidel,
  • Micha Pfeiffer

摘要

Purpose:

Markerless navigation in minimally invasive surgery is still an unsolved challenge. Many proposed navigation systems for minimally invasive surgeries rely on stereoscopic images, while in clinical practice oftentimes monocular endoscopes are used. Combined with the lack of automatic video-based navigation systems for prostatectomies, this paper explores methods to tackle both research gaps at the same time for robot-assisted prostatectomies.

Methods:

In order to realize a semi-automatic augmented reality overlay for navigated prostatectomy, the camera pose w.r.t. the prostate needs to be estimated. We developed a method where visual cues are drawn on top of the organ after an initial manual alignment, simultaneously creating matching landmarks on the 2D and 3D data. Starting from this key frame, the cues are then tracked in the endoscopic video. Both PnPRansac and differentiable rendering are then explored to perform 2D–3D registration for each frame.

Results:

We performed experiments on synthetic and in vivo data. On synthetic data differentiable rendering can achieve a median target registration error of 6.11 mm. Both PnPRansac and differentiable rendering are feasible methods for 2D–3D registration.

Conclusion:

We demonstrated a video-based markerless augmented reality overlay for navigated prostatectomy, using visual cues as an anchor.