Text-to-Image Person Re-identification via Optimal Transport-Based Priority Distribution
摘要
Text-to-Image Person Re-Identification (TIReid) is an important research direction at the intersection of person re-identification and cross-modal image retrieval. The core goal of this task is to retrieve person matching textual descriptions from huge image or video databases by using natural language descriptions as query conditions. Recognizing the differences in meaning between images and texts is a significant challenge due to the fundamental variations in their modes of expression. This is especially true for elements with low cross-modal relevance, which can hinder inter-modal alignment and make accurate matching challenging. This study introduces a TIReid approach, which leverages the Optimal matching Transport with Priority Distribution (OTPD) framework. It dynamically assesses the significance of various modal representations and adjusts their weights to thoroughly explore the correlations between visual and semantic features in the linguistic domain. Specifically, we first design a prioritization processing evaluation module, which uses predictive entropy to dynamically weight the importance of each view block and text markup as a way to improve the accuracy of prediction. Subsequently, we introduce an optimal transmission strategy that transforms the embedding distances between different image blocks and markers into an optimal transmission problem, which achieves cross-modal optimal matching inference by minimizing the structured distances between image views and text descriptions. This process effectively captures the fine-grained characteristics associated with the person’s identity, significantly improving re-identification accuracy. The method’s effectiveness and superiority are validated by its exceptional results across all three benchmark datasets, as demonstrated by the experimental findings.