EE-Head: emotion estimation for precise facial expression in NeRF head avatars
摘要
Reconstructing animatable avatars from images and videos plays an important role in virtual domains and immersive telepresence. Existing methods that combine parametric face models with neural radiance fields ignore the influence of subtle expressions on facial geometry. They have difficulty reconstructing photo-realistic avatars with accurate geometries and expressions. To tackle this problem, we propose EE-Head, which introduces a parametric face model with high-accurate facial expressions to neural radiance fields, enhancing the rendering quality of animatable avatars. Specifically, we first propose an emotion estimation algorithm, which utilizes an emotion consistency loss to encourage emotion similarity between input images and parametric face models. This algorithm can estimate a parametric face model with more accurate facial expressions, particularly in the lip region. Then, we propose a joint training method that optimizes a neural radiance field by adjusting the weights of the image and parametric models across different frames. Our optimization can adaptively adjust the impact of different frames on the neural radiance field, improving rendering quality. As a result, EE-Head can reconstruct photo-realistic and animatable avatars with accurate expressions from multi-view images and monocular dynamic videos. Extensive experiments on various subjects demonstrate that EE-Head outperforms SOTA methods in quantitative and qualitative rendering quality.