Neural face reenactment has emerged as a powerful tool in facial image synthesis and editing, enabling the transfer of poses and expressions between images while maintaining the source’s identity. Generative models have revolutionized face reenactment by utilizing their latent spaces for accurate manipulation of facial attributes. However, achieving high realism while preserving micro-surface details, such as subtle textural variations, remains a challenge. We present a streamlined framework for one-shot facial reenactment, utilizing StyleGAN2 and hypernetworks that tackles these issues by focusing on preserving source identity and fine details while effectively transferring the target’s pose and expression. We employ a novel style map generation technique that efficiently extracts the finer textural details and a fusion strategy for appearance, pose, and stylistic features ensuring high photorealism and adaptability. The hypernetwork dynamically refines generator weights, enhancing fine details like wrinkles and shadows while maintaining real-time feasibility. Extensive evaluations on the VoxCeleb1 dataset demonstrate superior performance compared to existing methods, achieving notable improvements in key metrics, including a Cosine Similarity (CSIM) score of 0.72 and a Fréchet Inception Distance (FID) of 26.7. Despite notable advances, limitations include challenges in reconstructing accessories and subtle background details. The proposed work contributes a scalable and effective approach to facial reenactment, suitable for applications in Augmented Reality and Virtual Reality (AR/VR), video production, and digital media, paving the way for further enhancements in identity-preserving image synthesis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Efficient One-Shot Face Reenactment Framework Using StyleGAN2

  • Sanjana Patil,
  • Pooja Gani,
  • Nitya Patil,
  • Anusri Kulkarni,
  • Uday Kulkarni,
  • Shashank Hegde

摘要

Neural face reenactment has emerged as a powerful tool in facial image synthesis and editing, enabling the transfer of poses and expressions between images while maintaining the source’s identity. Generative models have revolutionized face reenactment by utilizing their latent spaces for accurate manipulation of facial attributes. However, achieving high realism while preserving micro-surface details, such as subtle textural variations, remains a challenge. We present a streamlined framework for one-shot facial reenactment, utilizing StyleGAN2 and hypernetworks that tackles these issues by focusing on preserving source identity and fine details while effectively transferring the target’s pose and expression. We employ a novel style map generation technique that efficiently extracts the finer textural details and a fusion strategy for appearance, pose, and stylistic features ensuring high photorealism and adaptability. The hypernetwork dynamically refines generator weights, enhancing fine details like wrinkles and shadows while maintaining real-time feasibility. Extensive evaluations on the VoxCeleb1 dataset demonstrate superior performance compared to existing methods, achieving notable improvements in key metrics, including a Cosine Similarity (CSIM) score of 0.72 and a Fréchet Inception Distance (FID) of 26.7. Despite notable advances, limitations include challenges in reconstructing accessories and subtle background details. The proposed work contributes a scalable and effective approach to facial reenactment, suitable for applications in Augmented Reality and Virtual Reality (AR/VR), video production, and digital media, paving the way for further enhancements in identity-preserving image synthesis.