Time-consuming annotation of surgical video frames for pixel-level semantic segmentation hinders the development of intraoperative guidance systems for retinal microsurgery. This study evaluates prompting strategies within the Segment Anything Model 2 (SAM2) framework for generating dense segmentation masks. Three datasets were derived from videos of phantom retinal surgery, namely internal limiting membrane (ILM) peeling, by prompting SAM2 with either single-point prompts, bounding boxes, or manually annotated masks per object. Each dataset was used to train supervised segmentation models, namely U-Net, DeepLabV3+, TransUNet, and nnU-Net, under identical architectural and training settings. Performance was assessed using mean Intersection-over-Union and compared with manually annotated ground truth. Results indicate that bounding box prompts yield acceptable segmentation quality with minimal manual effort, whereas manually annotated mask prompts deliver superior pixel-level accuracy and consistency across sequences. Single-point prompts did not produce datasets that could assist with capturing fine instrument details. These findings support the use of SAM2 as a semi-automated annotation tool and offer comparative insight into prompting strategies for generating high-quality training labels in ophthalmic surgical video segmentation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Semi-automated Retinal Microsurgery Video Annotation with SAM2: Comparative Analysis of Prompt Strategies

  • Adriana Namour,
  • Oluwatosin Alabi,
  • Minghan Zhao,
  • Charalampos Komninos,
  • Tom Vercauteren,
  • Lyndon da Cruz,
  • Sebastien Ourselin,
  • Christos Bergeles

摘要

Time-consuming annotation of surgical video frames for pixel-level semantic segmentation hinders the development of intraoperative guidance systems for retinal microsurgery. This study evaluates prompting strategies within the Segment Anything Model 2 (SAM2) framework for generating dense segmentation masks. Three datasets were derived from videos of phantom retinal surgery, namely internal limiting membrane (ILM) peeling, by prompting SAM2 with either single-point prompts, bounding boxes, or manually annotated masks per object. Each dataset was used to train supervised segmentation models, namely U-Net, DeepLabV3+, TransUNet, and nnU-Net, under identical architectural and training settings. Performance was assessed using mean Intersection-over-Union and compared with manually annotated ground truth. Results indicate that bounding box prompts yield acceptable segmentation quality with minimal manual effort, whereas manually annotated mask prompts deliver superior pixel-level accuracy and consistency across sequences. Single-point prompts did not produce datasets that could assist with capturing fine instrument details. These findings support the use of SAM2 as a semi-automated annotation tool and offer comparative insight into prompting strategies for generating high-quality training labels in ophthalmic surgical video segmentation.