Learning-Based Baseline Method for Efficient Determination of Overlapping Image Pairs and Its Application On both Offline and Online SfM

In the current era of exponential growth in image data, the number of input images for SfM (Structure from Motion) tends to increase. Consequently, time efficiency has emerged as an insurmountable challenge for both large-scale offline SfM and online SfM. One of the enduring limitations in this domain is image matching. In this work, a baseline method consisting of a global feature fine-tuning solution based on learning-based models and a corresponding efficient indexing structure is proposed to fast identify overlapping image pairs (OIP). In particular, to guarantee sufficient retrieval performance regarding precision and recall, several popular backbone models as extractor of global features are fine tuned in a supervised manner with referenced overlapping image pairs, and data augmentation is employed by simulating various rotations to address rotation variance. In addition, with these fine-tuned global features, an efficient indexing structure constituted of hierarchical vocabulary tree is unsupervised trained to further improve the time efficiency of determining OIP. In the experiments, the results demonstrate that the fine-tuned global feature outperforms pre-trained models in terms of retrieval performance, and the proposed hierarchical vocabulary tree can significantly speed up the identification of OIP. Furthermore, when applied to offline and online SfM, it shows that time cost of image matching is remarkably reduced and real-time image matching performance is achieved, while still obtaining state-of-the-art SfM results.