Scholarlink

<p>Facial animation driven by speech signals plays a pivotal role in virtual reality, human-computer interaction, and entertainment. Traditional methods often struggle to capture the rich emotional and semantic context conveyed by the speaker’s voice, resulting in less engaging animations. This paper introduces Speech2Face, a framework that synthesizes high-fidelity facial movements conditioned on expressive speech inputs. The proposed approach utilizes EmotionBERT to extract semantic and emotional embeddings from speech transcripts, enabling the generation of temporally coherent and contextually appropriate facial expressions. Experiments on VOCASET, IEMOCAP, MEAD, and BIWI datasets demonstrate that Speech2Face improves emotional accuracy by up to 80.8%, naturalness, and synchronization between speech and facial movements compared to existing models. This framework advances multi-modal emotion-aware synthesis, offering an effective solution for creating engaging and realistic digital humans. Our code is available at <a href="https://github.com/rizwanchouhan/speech2face">https://github.com/rizwanchouhan/speech2face</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Emotionally expressive facial animation driven by EmotionBERT embeddings

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年7月1日

查看全文
  • Tahani Jaser Alahmadi,
  • Galiya Ybytayeva,
  • Harbi AlMahafzah,
  • Khalid J. Alzahrani,
  • Rizwan Abbas,
  • Masoud Alajmi,
  • Khalfalla Awedat

摘要

Facial animation driven by speech signals plays a pivotal role in virtual reality, human-computer interaction, and entertainment. Traditional methods often struggle to capture the rich emotional and semantic context conveyed by the speaker’s voice, resulting in less engaging animations. This paper introduces Speech2Face, a framework that synthesizes high-fidelity facial movements conditioned on expressive speech inputs. The proposed approach utilizes EmotionBERT to extract semantic and emotional embeddings from speech transcripts, enabling the generation of temporally coherent and contextually appropriate facial expressions. Experiments on VOCASET, IEMOCAP, MEAD, and BIWI datasets demonstrate that Speech2Face improves emotional accuracy by up to 80.8%, naturalness, and synchronization between speech and facial movements compared to existing models. This framework advances multi-modal emotion-aware synthesis, offering an effective solution for creating engaging and realistic digital humans. Our code is available at https://github.com/rizwanchouhan/speech2face.