<p>Constructing an effective common latent embedding by aligning the latent spaces of cross-modal variational autoencoders (VAEs) is a popular strategy for generalized zero-shot learning (GZSL). However, due to the lack of fine-grained instance-wise annotations, existing VAE methods can easily suffer from the posterior collapse problem. In this paper, we propose an innovative asymmetric VAE network by aligning enhanced feature representation (AEFR) for GZSL. Distinguished from general VAE structures, we designed two asymmetric encoders for visual and semantic observations and one decoder for visual reconstruction. Specifically, we propose a simple yet effective gated attention mechanism (GAM) in the visual encoder for enhancing the information interaction between observations and latent variables, alleviating the possible posterior collapse problem effectively. In addition, we propose a novel distributional decoupling-based contrastive learning (<i>D</i><sup>2</sup>-CL) to guide learning classification-relevant information while aligning the representations at the taxonomy level in the latent representation space. Extensive experiments on publicly available datasets demonstrate the state-of-the-art performance of our method. The source code is available at <a href="https://github.com/seeyourmind/AEFR">https://github.com/seeyourmind/AEFR</a>.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Aligning enhanced feature representation for generalized zero-shot learning

  • Zhiyu Fang,
  • Xiaobin Zhu,
  • Chun Yang,
  • Hongyang Zhou,
  • Jingyan Qin,
  • Xu-Cheng Yin

摘要

Constructing an effective common latent embedding by aligning the latent spaces of cross-modal variational autoencoders (VAEs) is a popular strategy for generalized zero-shot learning (GZSL). However, due to the lack of fine-grained instance-wise annotations, existing VAE methods can easily suffer from the posterior collapse problem. In this paper, we propose an innovative asymmetric VAE network by aligning enhanced feature representation (AEFR) for GZSL. Distinguished from general VAE structures, we designed two asymmetric encoders for visual and semantic observations and one decoder for visual reconstruction. Specifically, we propose a simple yet effective gated attention mechanism (GAM) in the visual encoder for enhancing the information interaction between observations and latent variables, alleviating the possible posterior collapse problem effectively. In addition, we propose a novel distributional decoupling-based contrastive learning (D2-CL) to guide learning classification-relevant information while aligning the representations at the taxonomy level in the latent representation space. Extensive experiments on publicly available datasets demonstrate the state-of-the-art performance of our method. The source code is available at https://github.com/seeyourmind/AEFR.