Scholarlink

Purpose <p>3D reconstruction in minimally invasive surgery (MIS) enables enhanced surgical guidance through improved visualisation, tool tracking, and augmented reality. However, traditional RGB-based keypoint detection and matching pipelines struggle with surgical challenges, such as poor texture and complex illumination. We investigate whether using snapshot hyperspectral imaging (HSI) can provide improved results on keypoint detection and matching surgical scenes.</p> Methods <p>We developed HyKey, a hyperspectral keypoint detection and description model made up of a hybrid 3D-2D convolutional neural network that jointly extracts spatial-spectral features from HSI. The model was trained using synthetic homographic augmentation and epipolar geometry constraints on a robotically acquired dual-camera RGB-HSI laparoscopic dataset of ex vivo organs with calibrated camera poses. We benchmarked performance against established RGB-based methods, including SuperPoint and ALIKE.</p> Results <p>Our HSI-based model outperformed RGB baselines on registered RGB frames, achieving 96.62% mean matching accuracy and 67.18% mean average accuracy at 10<InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(^\circ \)</EquationSource> <EquationSource Format="MATHML"><math> <mmultiscripts> <mrow /> <mrow /> <mo>∘</mo> </mmultiscripts> </math></EquationSource> </InlineEquation> on pose estimation, demonstrating consistent improvements across multiple evaluation metrics.</p> Conclusion <p>Integrating spectral information from an HSI cube offers a promising approach for robust monocular 3D reconstruction in MIS, addressing limitations of texture-poor surgical environments through enhanced spectral-spatial feature discrimination. Our model and dataset are available at <a href="https://github.com/alexsaikia/HyKey-Hyperspectral-Keypoint-Detection.">https://github.com/alexsaikia/HyKey-Hyperspectral-Keypoint-Detection.</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HyKey: hyperspectral keypoint detection and matching in minimally invasive surgery

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年4月27日

查看全文
  • Alexander Saikia,
  • Chiara Di Vece,
  • Zhehua Mao,
  • Sierra Bonilla,
  • Chloe He,
  • Joao Ramalhinho,
  • Tobias Czempiel,
  • Sophia Bano,
  • Danail Stoyanov

摘要

Purpose

3D reconstruction in minimally invasive surgery (MIS) enables enhanced surgical guidance through improved visualisation, tool tracking, and augmented reality. However, traditional RGB-based keypoint detection and matching pipelines struggle with surgical challenges, such as poor texture and complex illumination. We investigate whether using snapshot hyperspectral imaging (HSI) can provide improved results on keypoint detection and matching surgical scenes.

Methods

We developed HyKey, a hyperspectral keypoint detection and description model made up of a hybrid 3D-2D convolutional neural network that jointly extracts spatial-spectral features from HSI. The model was trained using synthetic homographic augmentation and epipolar geometry constraints on a robotically acquired dual-camera RGB-HSI laparoscopic dataset of ex vivo organs with calibrated camera poses. We benchmarked performance against established RGB-based methods, including SuperPoint and ALIKE.

Results

Our HSI-based model outperformed RGB baselines on registered RGB frames, achieving 96.62% mean matching accuracy and 67.18% mean average accuracy at 10 \(^\circ \) ∘ on pose estimation, demonstrating consistent improvements across multiple evaluation metrics.

Conclusion

Integrating spectral information from an HSI cube offers a promising approach for robust monocular 3D reconstruction in MIS, addressing limitations of texture-poor surgical environments through enhanced spectral-spatial feature discrimination. Our model and dataset are available at https://github.com/alexsaikia/HyKey-Hyperspectral-Keypoint-Detection.