HoloHand: enhancing real-time neural hand rendering with self-occlusion-aware appearance fields and depth-aware learning
摘要
In computer vision, real-time and photorealistic 3D hand rendering plays a critical role in human-computer interaction. However, existing Neural Radiance Field (NeRF)-based methods often struggle with the intricate geometry and self-occlusion of human hands, limiting both realism and efficiency. In this paper, we present HoloHand, a hybrid framework that enhances real-time neural hand rendering by integrating self-occlusion-aware appearance modeling and depth-aware learning. Built upon an explicit hand mesh representation, our method captures high-frequency appearance details via a neural implicit field while maintaining structural accuracy. To capture self-occlusion-induced appearance variations under dynamic hand poses—such as inter-finger shadowing—we introduce a self-occlusion-aware appearance field that learns to predict per-point irradiance attenuation from training data. Additionally, a depth-guided supervision loss is incorporated to improve geometry consistency, and a super-resolution module enables high-resolution output while maintaining inference speed within the real-time regime. Extensive experiments demonstrate that HoloHand achieves high-quality rendering results in real time, bridging the gap between performance and photorealism in neural hand rendering.