Open-World Object Detection Enhanced Image Matching
摘要
Image Matching (IM) aims to establish precise correspondences between visual elements (points, regions, structures) across multiple images. However, its performance severely degrades due to repetitive and dynamic objects appearing in images to be matched. In order to handle this problem, this paper proposes a novel image matching framework named open-world object detection-enhanced image matching (OWOD-IM). In OWOD-IM, YOLO-World, the state-of-the-art open-world object detector, is applied to extract rich object boxes and categories without specifying object labels. The detection results are exploited to improve the LightGlue matching performance as follows. At the initialization stage, the method removes false matches belonging to different categories and located on potential mobile objects (e.g., cars, people). Subsequently, an iteration between homography estimation and matching selection is activated as follows. Based on the refined matching results, a reliable inter-image homography transformation is first estimated. Secondly, for every pair of feature points, the vertices of their associated object boxes are projected onto the counterpart image via the transformation to generate mapped regions. Matches are preserved only if every point in a pair lies within the mapped regions of the respective image. The above process is repeated until the matching results become invariant or the maximum number of iterations is reached. Experimental results on the MegaDepth-1500 and HPatches datasets containing repetitive and dynamic objects demonstrate that OWOD-IM delivers significant performance improvements. OWOD-IM achieves 6–11% reduction in pose estimation error over existing sparse image matchers (e.g., OmniGlue, LightGlue). Compared to dense image matchers (e.g., LoFTR, CasMTR), it shows comparable performance, reducing the reprojection error within 5 px (pixels) by 5–10%, but enjoys much lower computational complexity.