<p>Visible-Infrared Person Re-Identification (VI-ReID) is a challenging task due to the large modality discrepancy between two modalities. In view of this, a two-level modality fusion compensation network (TLMFC-Net) is proposed by us. Concretely, we first design an image-level fusion compensation module (ILFCM) for attenuating the impact of modality discrepancy at the image level by interactively fusing the visible image and infrared image to create auxiliary images that contain rich modality complementary information. Then, we design a feature-level fusion compensation module (FLFCM) for further attenuating the impact of modality discrepancy at the feature level by interactively fusing the visible features, infrared features, auxiliary visible features, and auxiliary infrared features to obtain auxiliary visible deep features and auxiliary infrared deep features. Furthermore, we design a local feature extraction module (LFEM) to learn fine-grained local pedestrian representations. Extensive experiments conducted on SYSU-MM01 and RegDB reveal that our method reaches the current advanced level. Codes are available at <a href="https://github.com/421893969/TLMFC-Net.">https://github.com/421893969/TLMFC-Net.</a></p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A two-level modality fusion compensation network for visible-infrared person re-identification

  • Xinxin Hao,
  • Haishun Du,
  • Jieru Li,
  • Jiangtao Guo

摘要

Visible-Infrared Person Re-Identification (VI-ReID) is a challenging task due to the large modality discrepancy between two modalities. In view of this, a two-level modality fusion compensation network (TLMFC-Net) is proposed by us. Concretely, we first design an image-level fusion compensation module (ILFCM) for attenuating the impact of modality discrepancy at the image level by interactively fusing the visible image and infrared image to create auxiliary images that contain rich modality complementary information. Then, we design a feature-level fusion compensation module (FLFCM) for further attenuating the impact of modality discrepancy at the feature level by interactively fusing the visible features, infrared features, auxiliary visible features, and auxiliary infrared features to obtain auxiliary visible deep features and auxiliary infrared deep features. Furthermore, we design a local feature extraction module (LFEM) to learn fine-grained local pedestrian representations. Extensive experiments conducted on SYSU-MM01 and RegDB reveal that our method reaches the current advanced level. Codes are available at https://github.com/421893969/TLMFC-Net.