Style Separation and Content Recovery for Generalizable Sketch Re-identification and a New Benchmark
摘要
Existing research on Sketch-Photo Person Re-identification (Sketch Re-ID) has been conducted using a publicly available dataset comprising only 200 identities, yielding satisfactory retrieval results. However, the limited size and diversity of training datasets constrain the potential for performance enhancement in current Sketch Re-ID models. When applying general deep learning models to novel datasets characterized by unknown distributions, these models often overlook the impact of domain shifts resulting from distributional discrepancies among different datasets. This oversight frequently leads to performance deterioration and diminished generalization capabilities. To address this challenge, we introduce a large-scale, multi-source, multi-view synthetic sketch re-identification dataset, designated as MSMV-SK, which comprises 3,364 synthetic sketches associated with 1,364 identities. To develop a generalizable Sketch Re-ID model utilizing this dataset, we propose a cross-modal feature learning network aimed at extracting domain-invariant features. Specifically, we implement a Style Separation and Content Recovery (SSCR) module that mitigates the effects of style variations while reconstructing identity-related components from features derived from both sketch and photo modalities. Additionally, we formulate a Bi-constrained Optimization Loss (BOL) to enhance the discriminative power of identity-related features. Extensive experimental evaluations demonstrate that our model, trained on the synthetic MSMV-SK dataset, outperforms a model trained on the Mask1k dataset when assessed on the PKU dataset. Notably, our approach achieves a 4% improvement in Rank-1 accuracy and a 3.72% increase in mean Average Precision (mAP), underscoring its superior generalization capabilities. Dataset is publicly available at: https://github.com/Lulingyi01/MSMV-SK.git .