Construction industry plays an important role in the overall economic development of the country. Building infrastructure occupies a huge part of it. Ensuring the building's safety and enhancing the quality of the structure is a crucial thing for maintaining the good structural health of buildings. The traditional method, to detect defects in buildings, uses manual inspection, but is very time-consuming and can also lead to human errors. To address these issues, an automated defect detection model is developed using deep learning techniques based on You Only Look Once (YOLOv9) object detection algorithm. Additionally, the system is designed to be versatile as it identifies defects such as cracks, water seepage, mold, peeling paint and stairstep cracks. Notably, our model got the highest precision, recall and mean Average Precision (mAP) of 96.4%, 96.9% and 98.7%, respectively, for the class crack. This model is useful for early defect prevention as many issues can increase rapidly if left untreated. The automated system has the quality to improve inspection efficiency and reduce maintenance costs. It also promotes smarter and safer solutions to decrease unseen risks.