In Generalized Zero-Shot Learning (GZSL), we aim to recognize both seen and unseen categories using a model trained only on seen categories. In computer vision, this translates into a classification problem, where knowledge from seen categories is transferred to unseen ones by exploiting the relationships between visual features and available semantic information. However, learning this joint distribution is costly and requires one-to-one alignment with corresponding semantic information. We present a reversed conditional diffusion-based model (RevCD) that mitigates this issue by estimating the semantic density conditioned on visual inputs. Our RevCD model consists of a cross Hadamard-addition embedding of a sinusoidal time schedule, and a multi-headed visual transformer for attention-guided embeddings. The proposed approach introduces two key innovations. First, we apply diffusion models to zero-shot learning, a novel approach that exploits their strengths in capturing data complexity. Second, we reverse the process by approximating the semantic densities based on visual data, made possible through the classifier-free guidance of diffusion models. Empirical results demonstrate that RevCD achieves competitive performance compared to state-of-the-art generative methods on standard GZSL benchmarks. The complete code will be available on GitHub.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

RevCD: Reversed Conditional Diffusion for Generalized Zero-Shot Learning

  • William Heyden,
  • Habib Ullah,
  • Muhammad Salman Siddiqui,
  • Fadi Al Machot

摘要

In Generalized Zero-Shot Learning (GZSL), we aim to recognize both seen and unseen categories using a model trained only on seen categories. In computer vision, this translates into a classification problem, where knowledge from seen categories is transferred to unseen ones by exploiting the relationships between visual features and available semantic information. However, learning this joint distribution is costly and requires one-to-one alignment with corresponding semantic information. We present a reversed conditional diffusion-based model (RevCD) that mitigates this issue by estimating the semantic density conditioned on visual inputs. Our RevCD model consists of a cross Hadamard-addition embedding of a sinusoidal time schedule, and a multi-headed visual transformer for attention-guided embeddings. The proposed approach introduces two key innovations. First, we apply diffusion models to zero-shot learning, a novel approach that exploits their strengths in capturing data complexity. Second, we reverse the process by approximating the semantic densities based on visual data, made possible through the classifier-free guidance of diffusion models. Empirical results demonstrate that RevCD achieves competitive performance compared to state-of-the-art generative methods on standard GZSL benchmarks. The complete code will be available on GitHub.