Speech-driven 3D facial animation has achieved significant progress, producing increasingly realistic results. However, animations generated solely from audio input often lack expressive style. Current research predominantly focuses on style control using video or mesh sequences, neglecting scenarios relying exclusively on audio input, which results in less expressive outcomes. To address this limitation, we propose a novel method that employs a regional attention mechanism guided by a single identity vector, enabling the generation of stylistically consistent and highly expressive animations. Our approach customizes facial attention mask regions for individual speakers and incorporates two loss functions, \(L_{mask}\) and \(L_{normal}\) , to capture detailed style information through vertex movements and geometric variations. Extensive experiments and user studies demonstrate that our method outperforms existing approaches in qualitative and quantitative evaluations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speech-Driven 3D Facial Animation with Regional Attention for Style Capture

  • Bailin Yang,
  • Jiahao Pan,
  • Fangzhe Nan,
  • Jiajie Wu

摘要

Speech-driven 3D facial animation has achieved significant progress, producing increasingly realistic results. However, animations generated solely from audio input often lack expressive style. Current research predominantly focuses on style control using video or mesh sequences, neglecting scenarios relying exclusively on audio input, which results in less expressive outcomes. To address this limitation, we propose a novel method that employs a regional attention mechanism guided by a single identity vector, enabling the generation of stylistically consistent and highly expressive animations. Our approach customizes facial attention mask regions for individual speakers and incorporates two loss functions, \(L_{mask}\) and \(L_{normal}\) , to capture detailed style information through vertex movements and geometric variations. Extensive experiments and user studies demonstrate that our method outperforms existing approaches in qualitative and quantitative evaluations.