With the passage of time, the surface of ancient buildings has suffered damage due to natural erosion and human factors, and there is an urgent need for efficient and non-destructive detection methods to ensure the accuracy of their protection and restoration work. Different lighting conditions and shadows may affect the quality of the image, leading to blurred boundaries between damaged and normal areas, thereby affecting the accuracy of detection. Therefore, a surface damage detection method for ancient buildings based on artificial intelligence machine vision technology is proposed. Firstly, using image acquisition equipment, collect image data of ancient buildings and preprocess the images. Secondly, the surface damage annotation of ancient buildings based on dictionary learning significantly improves annotation efficiency through automated feature extraction, sparse representation, and the introduction of label consistency and multi label discriminative dictionary learning. Using image digitization processing technology to extract target image information features and comprehensively detect surface damage of ancient buildings. Finally, set warning thresholds for different types and degrees of injuries, and issue injury warnings based on the detection results. The experimental results show that after application, this method has significant advantages in \(F1\) score and high detection accuracy.