Abstract <p>The need for automatic solar panel defect detection is becoming more and more urgent due to the rising demand for new solar energy systems worldwide. Convolutional neural networks are very effective at solving the image classification problem across a wide range of domains. A model for detecting faults in solar panel surfaces was presented in this paper. The three primary stages of the suggested model are the data preparation stage, the ResNet50 hyperparameter optimization stage, and the image classification stage. Furthermore, a modified version of the light spectrum optimizer (MLSO) was introduced in this paper. In the optimization process of the original light spectrum optimizer, the proposed MLSO makes use of the outward search method. The ideal batch size and learning rate values are determined using MLSO. A publicly available dataset with six classes is employed. The proposed fault detection model can greatly improve the accuracy of solar panel defect detection, according to the experimental results. The model achieved an accuracy of 96.58%, a sensitivity of 98.36%, a specificity of 98.02%, and an F1-score of 96.72%. Furthermore, the results showed that, when compared to other well-known swarm optimization algorithms, the proposed MLSO produced the most effective results.</p>