Solar energy has become of specific interest since it is more affordable than any other energy source, causing less environmental hazard. With improved analytics and monitoring, anomalies in solar energy systems will be detected and resolved with speed, enhancing the overall prevention of failures, lengthening the periods of effective maintenance, and ensuring system dependability and efficiency. According to recent research, You Only Look Once (YOLO), one of the most advanced target detection algorithms available for multiple defects in solar panels, depends heavily on the training dataset for the best accuracy results. Therefore, its performance will be poor if the number of samples with defects is low. Using data augmentation to raise the quantity of training samples is a practical way to improve YOLO performance. This paper assesses the impact of various data augmentation techniques on the performance of the YOLOv5 model using precision, recall, and mAP50. The results reveal that the mAP50 of the YOLOv5 algorithm with the scaling data augmentation technique improves by 2.9% over that obtained with the original dataset. The combined YOLOv5 and scale data augmentation method gives precisions, recalls, and mAP50 scores attained for PV panel defect detection as 88.3%, 89.2%, and 89.9%, respectively.