Improving Visual Search in Medical Videos with Self-supervised Learning and Temporal Feature Integration
摘要
Traditional text based search methods, which depend on manual annotation, can be time-consuming and inconsistent. In contrast, content-based image retrieval (CBIR) presents a promising alternative by using visual content for search queries. This study introduces a new method to enhance the feature extraction of medical video frames by leveraging recent advancements in self-supervised learning (SSL). Our approach capitalizes on the repetitive nature of medical procedures and employs Drop Dynamic-Time-Warping (DropDTW) to integrate temporal information into the feature vectors, making them more suitable for searches and improving the accuracy of visual search tasks. Experiments on the Cholec80 dataset highlight the effectiveness of our method through both classification and search metrics, demonstrating its potential to enhance retrieval accuracy and efficiency without relying on extensive labeled datasets.