Visual-Semantic Dual Calibration Network for Zero-Shot Learning
摘要
Zero-shot learning (ZSL) is an important paradigm in machine learning research that aims to recognize unseen classes by leveraging shared semantic attributes between seen and unseen classes, training exclusively on the seen classes. Existing ZSL methods typically map visual features into semantic space for classification through visual-semantic alignment, and vice versa. However, a gap exists between artificially defined semantic prototypes and actual prototypes, and visual features may fail to highlight the key attribute-relevant features. This leads to inherent bias in both spaces, resulting in failed alignment and misclassification. In this paper, we propose a Visual-Semantic Dual Calibration Network (VSDCN) to address this bias in ZSL through a two-stage calibration process. During the training-stage calibration, we introduce a Visual Calibration Network (VCN) to enhance the visual space by incorporating semantic information. Additionally, a Semantic Calibration Network (SCN) is designed to calibrate the semantic space by refining semantic prototypes using visual information. During the testing-stage calibration, we jointly utilize the two complementary spaces for ZSL classification. Extensive experiments demonstrate that VSDCN achieves state-of-the-art results in ZSL. Compared to the latest transductive methods, VSDCN shows average improvements of 3.8%, 7.6%, and 3.4% in the harmonic mean on CUB, SUN, and AWA2, respectively.