Visible-infrared person re-identification (VI-ReID) matches pedestrian images across visible and infrared modalities. Some methods learn common semantic information at shallow layers and extract modality-specific and modality-shared features at deeper layers to achieve complementary information integration. This approach can cause semantic overlap between the both types of features, which impacts the learning of distinct semantic information. To address this issue, we propose a distinct semantic information learning framework. First, we introduce a Salient Specific Feature Mining Mechanism (SSFMM), which captures multi-scale salient information in modality-specific feature and enhances the discriminability. Second, we propose an Invariant Shared Feature Mining Mechanism (ISFMM), which reduces modality discrepancy and captures invariant information in modality-shared feature. Third, we propose a Semantic Overlap Mitigation Mechanism (SOMM) that captures global semantic information conforming to the symmetric structural properties of pedestrians through the visual state space block, while mitigating semantic overlap between modality-specific and modality-shared features by reducing semantic similarity. Our approach focuses on mining the distinct semantic information in modality-specific and modality-shared features. Comprehensive experiments on two datasets demonstrate the superiority of our method.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Learning Distinct Semantic Information in Modality-Specific and Modality-Shared Features for Visible-Infrared Person Re-identification

  • Min Jiang,
  • Huang Luo,
  • Jun Kong,
  • Ming Lu

摘要

Visible-infrared person re-identification (VI-ReID) matches pedestrian images across visible and infrared modalities. Some methods learn common semantic information at shallow layers and extract modality-specific and modality-shared features at deeper layers to achieve complementary information integration. This approach can cause semantic overlap between the both types of features, which impacts the learning of distinct semantic information. To address this issue, we propose a distinct semantic information learning framework. First, we introduce a Salient Specific Feature Mining Mechanism (SSFMM), which captures multi-scale salient information in modality-specific feature and enhances the discriminability. Second, we propose an Invariant Shared Feature Mining Mechanism (ISFMM), which reduces modality discrepancy and captures invariant information in modality-shared feature. Third, we propose a Semantic Overlap Mitigation Mechanism (SOMM) that captures global semantic information conforming to the symmetric structural properties of pedestrians through the visual state space block, while mitigating semantic overlap between modality-specific and modality-shared features by reducing semantic similarity. Our approach focuses on mining the distinct semantic information in modality-specific and modality-shared features. Comprehensive experiments on two datasets demonstrate the superiority of our method.