Cross-modal local fine-grained feature localization and alignment for text-to-image person re-identification
摘要
Text-to-image person re-identification (TIReID) aims to retrieve the target person images from the image gallery according to the specified text query. The key challenge lies in extracting the modality-aligned fine-grained representations. The existing methods extract modality-shared features by imposing consistency constraints on the features of text segments and image patches or performing cross-attention interaction among all words and image patches. However, these methods are prone to be disturbed by irrelevant information and overemphasize the most salient features, thus lacking the ability to extract diverse fine-grained features. To address these problems, we propose a Cross-modal local Fine-grained feature Localization and Alignment framework (CFLA), which captures diverse cross-modal co-saliency features between multiple text segments and images to improve the accuracy of text-image matching. Specifically, we first divide the original text into multiple text segments based on the sentence structure, while maintaining the local semantic integrity of each text segment. Then, we design a cross-modal bidirectional localization (CBL) module to mine multiple cross-modal local co-salient features by establishing cross-modal bidirectional mapping relationships. Finally, we introduce a cross-modal semantic alignment (CSA) module to further ensure the semantic consistency of cross-modal local co-salient features. Extensive experiments show that the proposed method achieves the best accuracy of Rank-1 on CUHK-PEDES, ICFG-PEDES, and RSTPReid datasets, which are 5.54%, 1.89%, and 5.20% higher than the most advanced methods, respectively.