SAKD: Self-attention Knowledge Distillation for Lifelong Person Re-identification
摘要
Lifelong person re-identification requires continuous learning from sequential, non-overlapping datasets while maintaining knowledge of previous identities. This task faces significant catastrophic forgetting, where adapting to new data degrades performance on previous ones. To address this issue, we propose a self-attention knowledge distillation (SAKD) framework that extracts knowledge based on the self-attention structure and enhances the base performance using the strong semantic guidance from contrastive learning. Specifically, our approach (1) designs a self-attention extraction mechanism that distills the self-attention region information from the later layers of the Vision Transformer (ViT) and weights the knowledge based on its credibility to enforce consistency in salient region identification. (2) Leverages text descriptions from the pretrained text encoder to guide training and comprehensively improves model performance through strong semantic alignment in text-image contrastive learning. Extensive experiments on 12 benchmark datasets and comparisons with 8 representative models demonstrate our method’s high performance and effectiveness in knowledge retention, showcasing cross-modal learning’s potential for lifelong ReID in open-world scenarios.