SFPFR: Self-supervised Facial Paralysis Face Reconstruction Under Few Views
摘要
3D face reconstruction methods exhibit significant limitations when applied to pathological cases such as facial paralysis, due to inherent challenges including asymmetric motion and non-linear muscle dynamics. To address these gaps, we propose SFPFR, a self-supervised framework for facial paralysis 3D face reconstruction leveraging 1-3 viewpoints. We first propose a self-supervised learning paradigm integrating reconstruction loss, multi-view consistency loss, and a Mamba-based temporal loss to reconstruct 3D face without ground-truth; then, a partitioned dynamic fusion module that adaptively weights multi-view features ensuring precise geometric reconstruction and pathological detail preservation; last, we propose FPD-100, the first multi-view video dataset for facial paralysis, comprising 30,000 frames from 100 patients of 3 views. Extensive experiments validate SFPFR’s superiority, achieving state-of-the-art PSNR (27.74) and FID (37.13). It enables clinical applications in severity assessment, rehabilitation monitoring, and treatment planning, while the dataset and code will be open-sourced to catalyze research in pathological facial analysis.