Considering the high expenses, low efficiency, and inadequate precision prevalent in the current realm of building structure damage detection, this paper presents a building structure damage detection system founded on deep learning. The system's objective is to attain efficient and precise damage detection via artificial intelligence technology, thus fulfilling the requirements of large-scale and high-precision detection. Initially, this paper assembles a high-quality dataset by amassing a substantial quantity of building structure image data. These data encompass a multitude of damage types and severities, laying a robust foundation for subsequent model training. Secondly, this paper employs acoustic wave detection to formulate a damage detection model. Through numerous rounds of training and optimization, the model can precisely identify the damage type and location within the building structure. The experimental findings indicate that, in regard to data, data augmentation techniques effectively enlarge the dataset's size and enhance the model's generalization capability. Abundant and diverse data samples empower the model to learn a broader spectrum of damage features, consequently demonstrating superior performance on the test set. In terms of the model architecture, the enhanced U-Net +  + and the one-dimensional convolutional neural network based on VGG-16 are optimized for image and sound wave data respectively, and can efficiently extract features from the data. A rational network structure and appropriate parameter settings serve as crucial assurances for the model's performance.