The problem of 3D Reconstruction and restoration of Indian heritage sites is a classical 3D computer vision problem which has been addressed in the Indian vision and graphics research communities. The major focus of these approaches has been to reconstruct a point cloud model of the heritage sites from multiple colored images captured from different viewpoints. This essentially requires solving the structure from motion problem to be solved. This becomes a challenging problem to solve for large scale architectural sites. With the emergence of 3D scanners, we can obtain partial 3D scans of a site from different viewpoints. Then, the main problem to solve is to fuse these noisy and partial 3D scans to obtain the complete 3D model of the underlying surface of the heritage site. The problem of reconstructing the complete 3D model from the partial 3D scan can be solved using classical approaches as well as learning based approaches. Using machine learning based approaches in the context of Indian heritage may suffer from the limited training dataset availability. Also, the order of points representing the 3D model of such sites can be in millions. Therefore, reconstructing such high resolution 3D models using neural network may demand high-end parallel computing facility.