Surgical scene reconstruction from endoscopic video is crucial for many applications in computer- and robot-assisted surgery. However, existing methods primarily focus on soft tissue deformation while often neglecting the dynamic motion of surgical tools, limiting the completeness of the reconstructed scene. To bridge the aforementioned research gap, we propose \(\textbf{T}^{2}\) GS, a novel and efficient surgical scene reconstruction framework that enables efficient spatio-temporal modelling of both deformable tissues and dynamically interacting surgical tools. \(\text {T}^{2}\) GS leverages Gaussian Splatting for dynamic scene reconstruction, and it integrates a recent tissue deformation modelling technique while most importantly, introduces a novel efficient tool motion model (ETMM). At its core, ETMM disambiguates the modelling process of tool’s motion as global trajectory modelling and local shape-change modelling. We additionally propose pose-informed pointcloud fusion (PIPF), holistically initialized of tools’ gaussians for improved tool motion reconstruction. Extensive experiments on public datasets demonstrate \(\text {T}^{2}\) GS’s superior performance for comprehensive endoscopic scene reconstruction compared to previous methods. Moreover, as we specifically design our method with efficiency in concern, \(\text {T}^{2}\) GS also showcases promising reconstruction efficiency (3mins) and rendering speed (71fps), highlighting its potential for intraoperative applications. Our code is available at https://gitlab.com/nct_tso_public/ttgs .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

\(\text {T}^{2}\) GS: Comprehensive Reconstruction of Dynamic Surgical Scenes with Gaussian Splatting

  • Jinjing Xu,
  • Chenyang Li,
  • Peng Liu,
  • Micha Pfeiffer,
  • Liwen Liu,
  • Reuben Docea,
  • Martin Wagner,
  • Stefanie Speidel

摘要

Surgical scene reconstruction from endoscopic video is crucial for many applications in computer- and robot-assisted surgery. However, existing methods primarily focus on soft tissue deformation while often neglecting the dynamic motion of surgical tools, limiting the completeness of the reconstructed scene. To bridge the aforementioned research gap, we propose \(\textbf{T}^{2}\) GS, a novel and efficient surgical scene reconstruction framework that enables efficient spatio-temporal modelling of both deformable tissues and dynamically interacting surgical tools. \(\text {T}^{2}\) GS leverages Gaussian Splatting for dynamic scene reconstruction, and it integrates a recent tissue deformation modelling technique while most importantly, introduces a novel efficient tool motion model (ETMM). At its core, ETMM disambiguates the modelling process of tool’s motion as global trajectory modelling and local shape-change modelling. We additionally propose pose-informed pointcloud fusion (PIPF), holistically initialized of tools’ gaussians for improved tool motion reconstruction. Extensive experiments on public datasets demonstrate \(\text {T}^{2}\) GS’s superior performance for comprehensive endoscopic scene reconstruction compared to previous methods. Moreover, as we specifically design our method with efficiency in concern, \(\text {T}^{2}\) GS also showcases promising reconstruction efficiency (3mins) and rendering speed (71fps), highlighting its potential for intraoperative applications. Our code is available at https://gitlab.com/nct_tso_public/ttgs .