HoloPointNet presents a novel deep-learning framework for 3D point cloud holography. Generally, computer-generated holography (CGH) methods typically rely on stacked 2D slices and suffer from inefficiencies. These 2D slices often contain empty regions in natural 3D scenes or are intentionally sparse in applications like holographic optogenetics. This results in excessive memory consumption and increased processing latency. In contrast, HoloPointNet directly processes 3D point cloud data using a concatenation-based feature extractor, followed by hierarchical upsampling and wavefront reconstruction modules, eliminating redundant spatial regions and improving efficiency. This design allows for the direct mapping of point cloud data to phase modulations for spatial light modulators (SLMs). By employing a structured convolutional feature transformation pipeline, HoloPointNet enables hierarchical refinement of spatial embeddings, enhancing feature encoding accuracy. HoloPointNet offers the capability to generate multiplane holograms, effectively addressing the complexities of 3D volumetric data. This capability, combined with fast inference times, enables real-time holography for applications such as optogenetics. The code is available at https://github.com/AnkitAmrutkar/HoloPointNet.git .