<p>Person re-identification (Re-ID) aims at matching the same individuals across different camera views. In the real scenes, existing Re-ID methods always suffer from various degradation factors, including motion blur, low illumination and occlusion contained in frames, leading to the loss of identity-discriminating features. Recently, neuromorphic vision sensors (event cameras) have been considered a potential solution to improve Re-ID performance, because they offer numerous advantages over standard cameras (high temporal resolution and high dynamic range) and can accurately record the regions of pedestrians in the degraded environments based on the scene dynamics. In this work, we propose an event-guided multi-degradation learning framework, which explores the complementary information of event streams to extract identity-related robust features in real-world degradations. Specifically, to properly extract the appearance information contained in events, we design a deformable spiking neural network to encode events into the spiking form, followed by a cross feature alignment module to fuse the appearance cues from frames and events to enhance identity representation learning. To fully utilize the dynamic characteristic of events, we firstly extract the key-point of pedestrians from events, then propose a spatial-temporal fusing module for effective fine-grained spatial and temporal feature fusion with frame features. Finally, all features will pass through a pyramid aggregation structure to facilitate cross-scale information propagation for capturing hierarchical spatial-temporal dependencies. Extensive experiments demonstrate the superiority of our framework, e.g., on LIPS dataset, it achieves 5.1% mAP improvement over the state-of-the-art method.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Learning Robust Event-Guided Representations for Person Re-Identification

  • Chengzhi Cao,
  • Xueyang Fu,
  • Senyan Xu,
  • Chengjie Ge,
  • Kunyu Wang,
  • Zheng-Jun Zha

摘要

Person re-identification (Re-ID) aims at matching the same individuals across different camera views. In the real scenes, existing Re-ID methods always suffer from various degradation factors, including motion blur, low illumination and occlusion contained in frames, leading to the loss of identity-discriminating features. Recently, neuromorphic vision sensors (event cameras) have been considered a potential solution to improve Re-ID performance, because they offer numerous advantages over standard cameras (high temporal resolution and high dynamic range) and can accurately record the regions of pedestrians in the degraded environments based on the scene dynamics. In this work, we propose an event-guided multi-degradation learning framework, which explores the complementary information of event streams to extract identity-related robust features in real-world degradations. Specifically, to properly extract the appearance information contained in events, we design a deformable spiking neural network to encode events into the spiking form, followed by a cross feature alignment module to fuse the appearance cues from frames and events to enhance identity representation learning. To fully utilize the dynamic characteristic of events, we firstly extract the key-point of pedestrians from events, then propose a spatial-temporal fusing module for effective fine-grained spatial and temporal feature fusion with frame features. Finally, all features will pass through a pyramid aggregation structure to facilitate cross-scale information propagation for capturing hierarchical spatial-temporal dependencies. Extensive experiments demonstrate the superiority of our framework, e.g., on LIPS dataset, it achieves 5.1% mAP improvement over the state-of-the-art method.