Inspection and maintenance of pavements are crucial for ensuring the safety and efficiency of urban transportation systems. Traditionally, pavement crack assessment heavily relied on visual inspections by humans, which is labor-intensive and time-consuming. Computer vision-assisted methods have improved the processes of damage recognition; however, these methods still require considerable manual involvement, including tasks such as data pre- and post-processing. Specifically, due to the complicated features of cracks and variations in image capture conditions, the types and boundaries of cracks may not be properly recognized, thereby requiring manual efforts to denoise and refine the results. This research proposes an automated framework to segment pavement cracks and translate local image data into global point cloud representations for supporting future quantitative severity analysis. The framework encompasses three primary steps: (1) conducting imagery-based pavement reality capture using unmanned aerial vehicles (UAVs); (2) developing deep learning-based segmentation models along with a macro–micro segmentation strategy to delineate pavement areas and crack instances; and (3) implementing 2D-to-3D projection approach to reconstruct pavement scenes and map among 2D crack features and 3D point cloud structures, thereby converting local segmentation results into a global 3D spatial scene. The feasibility and performance of the framework are validated using a real-world road pavement scene in Macau SAR, China, where 3D point cloud classification of cracks achieved an F1-score of 0.66. This framework enhances recognition accuracy of UAV-captured imagery and acquires cracks’ locations and dimensions for qualitative pavement condition assessment.