Scholarlink

<p>Dynamic Facial Expression Recognition (DFER) in video sequences presents significant challenges due to the subtle nuances between expressions and inherent dataset imbalances. This study proposes a novel approach leveraging large-scale contrastive visual language pre-training (VLP) models, specifically CLIP and ALIGN, to address these issues. We introduce two key innovations: (1) a concept-guided language-visual feature learning method and (2) a concept-guided sample rebalance technique. The former improves upon traditional text-to-image matching by selecting fine-grained text concepts (e.g., smiling lips, squinting eyes) and constructing corresponding visual concepts using CLIP. This method achieves precise fusion through visual concept backward lookup during training, enhancing semantic information in visual representations. The latter addresses dataset imbalance by synthesizing additional samples for underrepresented expressions and reducing redundancy in overrepresented categories. Our approach is evaluated on three benchmark datasets: DFEW, MAFW, and FERV39k. Results demonstrate significant improvements in both accuracy and robustness of DFER systems compared to existing methods.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

CLVSR: Concept-Guided Language-Visual Feature Learning and Sample Rebalance for Dynamic Facial Expression Recognition

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年1月24日

查看全文
  • Zichen Liang,
  • Haiying Xia,
  • Yumei Tan,
  • Shuxiang Song

摘要

Dynamic Facial Expression Recognition (DFER) in video sequences presents significant challenges due to the subtle nuances between expressions and inherent dataset imbalances. This study proposes a novel approach leveraging large-scale contrastive visual language pre-training (VLP) models, specifically CLIP and ALIGN, to address these issues. We introduce two key innovations: (1) a concept-guided language-visual feature learning method and (2) a concept-guided sample rebalance technique. The former improves upon traditional text-to-image matching by selecting fine-grained text concepts (e.g., smiling lips, squinting eyes) and constructing corresponding visual concepts using CLIP. This method achieves precise fusion through visual concept backward lookup during training, enhancing semantic information in visual representations. The latter addresses dataset imbalance by synthesizing additional samples for underrepresented expressions and reducing redundancy in overrepresented categories. Our approach is evaluated on three benchmark datasets: DFEW, MAFW, and FERV39k. Results demonstrate significant improvements in both accuracy and robustness of DFER systems compared to existing methods.