Purpose <p>Predicting surgical time completion helps streamline surgical workflow and OR utilization, enhancing hospital efficacy. When time prediction is based on interventional video of the surgical site, time predictions may correlate with technical proficiency of the surgeon because skill is a useful proxy of completion time. To understand features that are predictive of surgical time in surgical site video, we develop prototype-like visual explanations, making them applicable to video sequences.</p> Methods <p>We introduce an interpretable method for predicting surgical duration by identifying prototype-like patterns within egocentric video of the surgical site. Unlike conventional image-based prototype models that generate patch-based prototypes, our method extracts video-based explanations tied to segments of surgical videos with similar time deviation patterns. We achieve this by comparing the principal components of feature representation differences at various time points in the predictions. To effectively capture long-range dependencies in the prediction task, we employ an informer as the primary predictive model.</p> Results <p>This model is applied to a dataset of 42 point-of-view craniotomy videos, collected under an approved IRB protocol. On average, our interpretable model performs better than the baseline models in surgical time completion.</p> Conclusion <p>Our approach not only contributes to the interpretability of surgical time predictions but also takes full advantage of the detailed information provided by surgical video data.</p>