Nritya3D: Monocular Video-Driven Expressive 3D Human Mesh Reconstruction
摘要
In this paper, we propose Nritya3D: a two-stage framework designed to recover expressive 3D human mesh from monocular video as input, towards crafting 3D models for expressive dance forms. Indian classical dance forms such as Bharatanatyam feature intricate hand mudras (poses) and rich expressiveness that challenge existing single-view 3D human reconstruction methods. While recent methods regress pose parameters of expressive 3D human mesh represented as Expressive Skinned Multi-Person Linear (SMPL-X) model, they often struggle to capture complex hand movements. Towards this, we propose to decouple estimating body pose and hand pose of SMPL-X model separately. Naïve combination of both parameters tends to overlook actual wrist orientation and can lead to the reconstruction of conflicting poses. To tackle this issue, we introduce a tailored optimization for the combined SMPL-X parameters and propose a skeletal loss, which effectively captures the complexities of hand movements while conforming to overall body pose. We evaluate Nritya3D against benchmark dataset and methodologies, demonstrating quantitative performance and superior qualitative fidelity in reconstructing detailed mudras.