<p>Video-based person re-identification involves retrieving and matching video sequences of persons across non-overlapping cameras in different scenes and viewpoints. Current methods focus on single-domain tasks, resulting in models that excel in specific domains but perform poorly when facing domain shifts. As there are no publicly available methods for single-source domain generalization in video-based person re-identification, this issue needs to be addressed. To overcome the limitations of traditional convolutional neural networks in capturing long-range temporal dependencies, this paper proposes a domain-generalizable video-based person re-identification method (ViDG-CLIP) using the multi-modal large model CLIP. The method leverages the image encoder of CLIP to extract video sequence features, updated through an instance-adaptive prompting module. A random grayscale processing module (i.e., Color Regularization) enhances data diversity by applying random graying to video sequences. Additionally, a spatio-temporal information fusion module combines temporal fusion attention and spatial aggregation attention to better capture long-range spatio-temporal dependencies. Extensive experiments on five datasets show that the proposed method outperforms state-of-the-art approaches in generalization and that using a Transformer architecture offers superior domain generalization performance compared to convolutional neural networks.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Exploring CLIP with color regularization for single-source generalizable video-based person Re-ID

  • Lu Yu,
  • Ming Gao,
  • Lei Qi,
  • Wankou Yang

摘要

Video-based person re-identification involves retrieving and matching video sequences of persons across non-overlapping cameras in different scenes and viewpoints. Current methods focus on single-domain tasks, resulting in models that excel in specific domains but perform poorly when facing domain shifts. As there are no publicly available methods for single-source domain generalization in video-based person re-identification, this issue needs to be addressed. To overcome the limitations of traditional convolutional neural networks in capturing long-range temporal dependencies, this paper proposes a domain-generalizable video-based person re-identification method (ViDG-CLIP) using the multi-modal large model CLIP. The method leverages the image encoder of CLIP to extract video sequence features, updated through an instance-adaptive prompting module. A random grayscale processing module (i.e., Color Regularization) enhances data diversity by applying random graying to video sequences. Additionally, a spatio-temporal information fusion module combines temporal fusion attention and spatial aggregation attention to better capture long-range spatio-temporal dependencies. Extensive experiments on five datasets show that the proposed method outperforms state-of-the-art approaches in generalization and that using a Transformer architecture offers superior domain generalization performance compared to convolutional neural networks.