Zero-shot learning (ZSL) addresses the challenge of recognizing unseen classes by leveraging semantic relationships between seen and unseen categories. Traditional approaches often rely on linear projection functions and manually defined attribute signatures, which constrain model expressiveness and scalability. This study introduces a deep non-linear visual-semantic mapping framework, designed to capture complex relationships between image features and class semantics. Semantic embeddings are automatically derived from natural language descriptions using pre-trained BERT representations, eliminating the need for expert-annotated attributes. To improve discriminative power, a dual-objective loss function is employed, combining cosine similarity with a contrastive margin-based component to enforce both alignment and separation in the embedding space. Experimental results on three standard benchmarks as AwA, aPY, and SUN, demonstrate consistent improvements over strong baselines in unseen class accuracy. These findings highlight the effectiveness of integrating language-driven semantics and deep alignment mechanisms for scalable and robust zero-shot learning.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Enhancing Zero-Shot Learning with Non-linear Visual-Semantic Mapping and Dual Loss

  • Thi Thuy Le,
  • Hung Cuong Tran,
  • Dinh Cong Nguyen

摘要

Zero-shot learning (ZSL) addresses the challenge of recognizing unseen classes by leveraging semantic relationships between seen and unseen categories. Traditional approaches often rely on linear projection functions and manually defined attribute signatures, which constrain model expressiveness and scalability. This study introduces a deep non-linear visual-semantic mapping framework, designed to capture complex relationships between image features and class semantics. Semantic embeddings are automatically derived from natural language descriptions using pre-trained BERT representations, eliminating the need for expert-annotated attributes. To improve discriminative power, a dual-objective loss function is employed, combining cosine similarity with a contrastive margin-based component to enforce both alignment and separation in the embedding space. Experimental results on three standard benchmarks as AwA, aPY, and SUN, demonstrate consistent improvements over strong baselines in unseen class accuracy. These findings highlight the effectiveness of integrating language-driven semantics and deep alignment mechanisms for scalable and robust zero-shot learning.