Self-supervised single-view 3D point cloud reconstruction through GAN inversion

Recent single-view reconstruction methods have sought to reconstruct 3D point clouds from images and corresponding silhouette collections alone. However, merely utilizing input images as supervision without any auxiliary methods amplifies the matching ambiguity. To address this issue, we propose a self-supervised 3D point cloud reconstruction method based on generative adversarial network (GAN) inversion. Three novel components are introduced to solve the intrinsic challenges of cross-dimensional inversion. First, we develop a uniform loss to enhance the uniformity of the point clouds generated by the GAN. Second, we devise a coarse-to-fine differentiable point cloud renderer to facilitate accurate projections. Third, we design a pseudo ground-truth pose predictor that can estimate the precise viewpoints of the input images. Experimental results on both synthetic datasets and real-world datasets demonstrate that our approach outperforms existing state-of-the-art 2D supervised reconstruction methods and is comparable to 3D supervised approaches.