In this study, we focus on addressing the problem of matching passengers to their checked luggage when tags are lost during air travel. To use computer vision’s image retrieval technology to solve this issue, we first collected a dataset and a testing set that reflect real-world scenarios, based on the data characteristics, we propose the Effective Query Ratio (EQR), to measure the accuracy of retrieval results. To improve the EQR on the testing set, we optimized the system framework in two ways. First, we filtered out background interference by using a segmented encoder, and then introduced a mask-augmented segmentation encoder to prevent the loss of edge information in semantically segmented images, thereby better extracting image features and improving the query ratio. Second, we introduced a multi-image query mode to integrate information from multiple query images, further enhancing the query ratio. Experimental results show that our system has a 55.9% chance of including the corresponding passenger in the top 5 retrieval results, an 80.1% chance in the top 15, and a 96.6% chance in the top 30, demonstrating its potential application and effectiveness in retrieving luggage corresponding to passengers within a certain range. This study provides a new solution for matching lost luggage with passengers in the aviation industry, expanding the practical application scenarios of computer vision’s image retrieval technology in real life.