A human–AI collaborative framework for surgical field-of-view adjustment: design and experimental validation
摘要
The goal of robotic endoscope holders is to replace human assistants by enabling optimal field of view (FoV) adjustment while ensuring stable visualization during minimally invasive surgery (MIS). However, existing systems struggle to achieve this goal, relying either on autonomous strategies that fail to interpret the surgical true intent or on detailed commands that increase the surgeon’s interaction burden. Drawing inspiration from clinical decision-making processes, we introduce a novel Human–AI collaborative framework for surgical FoV adjustment (HIC-FoV) to address these limitations. This framework requires only concise voice commands from the surgeon, while leveraging AI to interpret the surgical scene and generate intent-aligned FoV adjustments. At the core of HIC-FoV is a novel multi-dimensional weighting mechanism, which evaluates the surgical scene based on directional and spatial congruence with commands, contextual significance, and perceptual confidence to enable efficient FoV optimization. Experiments conducted on peg transfer and simulated cholecystectomy tasks demonstrated that HIC-FoV achieved performance comparable to manual control (completion time: