Semi-automated Retinal Microsurgery Video Annotation with SAM2: Comparative Analysis of Prompt Strategies
摘要
Time-consuming annotation of surgical video frames for pixel-level semantic segmentation hinders the development of intraoperative guidance systems for retinal microsurgery. This study evaluates prompting strategies within the Segment Anything Model 2 (SAM2) framework for generating dense segmentation masks. Three datasets were derived from videos of phantom retinal surgery, namely internal limiting membrane (ILM) peeling, by prompting SAM2 with either single-point prompts, bounding boxes, or manually annotated masks per object. Each dataset was used to train supervised segmentation models, namely U-Net, DeepLabV3+, TransUNet, and nnU-Net, under identical architectural and training settings. Performance was assessed using mean Intersection-over-Union and compared with manually annotated ground truth. Results indicate that bounding box prompts yield acceptable segmentation quality with minimal manual effort, whereas manually annotated mask prompts deliver superior pixel-level accuracy and consistency across sequences. Single-point prompts did not produce datasets that could assist with capturing fine instrument details. These findings support the use of SAM2 as a semi-automated annotation tool and offer comparative insight into prompting strategies for generating high-quality training labels in ophthalmic surgical video segmentation.