A Bi-directional Optimization Network for De-obscured 3D High-Fidelity Face Reconstruction

3D detailed face reconstruction based on monocular images aims to reconstruct a 3D face from a single image with rich face detail. The existing methods have achieved significant results, but still suffer from inaccurate face geometry reconstruction and artifacts caused by mistaking hair for wrinkle information. To address these problems, we propose a bi-directional optimization network for de-obscured 3D high-fidelity surface reconstruction. Specifically, our network is divided into two stages: face geometry fitting and face detail optimization. In the first stage, we design a global and local bi-directional optimized feature extraction network that uses both local and global information to jointly constrain the face geometry and ultimately achieve an accurate 3D face geometry reconstruction. In the second stage, we decouple the hair and the face using a segmentation network and use the distribution of depth values in the facial region as a prior for the hair part, after which the FPU-net detail extraction network we designed is able to reconstruct finer 3D face details while removing the hair occlusion problem. With only a small number of training samples, extensive experimental results on multiple evaluation datasets show that our method achieves competitive performance and significant improvements over state-of-the-art methods.