Unified Token Representation and Accurate Attribute Prediction for Generalized Zero-Shot Learning
摘要
Generalized zero-shot enables the recognition of unseen classes by transferring semantic knowledge from seen classes, which is achieved by learning the latent relationship between visual features and attribute features. Previous works simply align the feature of an image with its associated attribute descriptions. However, direct alignment of image and text modalities often results in spurious visual-semantic correlations, owing to the inherent differences in semantic levels and granularity between the features extracted from image samples and text representations. To solve this situation, we introduce a Finite Discrete Token Module (FDTM) across the two modalities, using unified token representations to encode images and texts, achieving consistency between image and text embeddings at both the granularity and semantic levels. In addition, we propose a Semantic Correlation Intervention Module (SCIM) analyzes the relationships between different attribute response values in the attention mechanism, making the predictions closer to the ground truth and improving attribute prediction accuracy. The proposed method is evaluated on three ZSL benchmarks, and the results demonstrate the feasibility of our proposed method.