Recent advances in digital technologies and automated analysis techniques applied to historic buildings have shown significant benefits in efficiently collecting and interpreting data for conservation activities. Close-range photogrammetry has become a valuable tool for detecting damage in historic buildings due to its non-invasive nature, which allows for the identification of issues while preserving the building’s structure. In particular, detecting and measuring damage on historic tiled surfaces is essential for the maintenance and protection of these buildings. However, current visual inspection methods are time-consuming and labor-intensive. This study proposes an automated inspection system that uses a trained and validated convolutional neural network for classifying degradation phenomena based on images acquired through photogrammetric surveys. The detection, segmentation, and quantification strategy for degradation phenomena relies on deep learning techniques to automatically detect and measure damage affecting historic tiles. Additionally, the Historic Building Information Model (HBIM) serves as an information repository by including semantic and graphical components for comprehensive documentation management in cultural heritage conservation and restoration field. The results highlight the potential of these techniques for detecting heritage damage, supporting decision-makers in planning recovery and maintenance interventions.