Structural health monitoring of buildings has always been a key area to ensure public safety and long-term building sustainability. However, traditional monitoring methods are often limited by their limited ability to identify complex structures and potential damage patterns, which motivates us to explore and innovatively apply deep learning technology to this field. After an in-depth analysis of the challenges of traditional monitoring methods in the face of diverse building structures and changing environments, we emphasize the distinctive benefits of utilizing deep learning for the extraction and analysis of intricate images, showcasing its unparalleled advantages in handling complex visual features. Based on this concept, we improved the YOLOv4 algorithm and obtained a new building health detection method. This method aims to make full use of the efficient target recognition and positioning capabilities of the YOLOv4 algorithm to accurately detect potential problems in building structures. By optimizing the data collection and annotation process, we established a reliable and fully annotated dataset, thereby improving the high accuracy and robustness of the model, and providing a solid foundation for model training and verification. In addition, we introduce an adaptive parameter adjustment method based on reinforcement learning Chen (Ain Shams Eng J 102621, 2024), which enables the model to better adapt to complex architectural environments and demonstrate reliable performance in different scenarios. Through a series of detailed experimental verifications, we concluded that the improved YOLOv4 building health detection method has significantly improved in accuracy and practicality, which provides a new perspective and useful practice for the technological development in the field of building structural health monitoring. It also provides strong support for the future application of deep learning in building safety monitoring, which is of great significance to improving the safety of building structures.