Detecting conformal coating defects, particularly de-wetting and capillary defects, is essential because these defects severely affect the coating's thickness uniformity and protective capabilities on Printed Circuit Boards (PCBs). Accurate defect diagnosis is key to optimizing the conformal coating application process. However, distinguishing between de-wetting and capillary defects presents significant challenges due to their visual similarities and subtle differences compared to other defect types. These complexities are further amplified under typical ultraviolet (UV) illumination conditions used during inspections. Therefore, automating defect diagnosis is vital to accurately identify and classify these defects, facilitating the effective optimization of coating process parameters. This paper presents a customized Faster Region-Based Convolutional Neural Network (Faster R-CNN) model for automated defect detection and classification in PCB assembly. The model incorporates ResNet-152 as the backbone to improve feature localization, while Region Proposal Network (RPN) modifications enhance the detection of small defects. Data augmentation techniques, including contrast-based preprocessing, are applied to enhance feature extraction and improve defect visibility. Additionally, increased data annotation is used to address class imbalance, ensuring better defect representation in the training process. High-resolution PCB images, combined with these enhancements, enable the system to handle challenging lighting conditions and detect subtle defect patterns with higher accuracy. Experimental results demonstrate strong performance, with recall rates of 98.53% for capillary defects and 95.49% for de-wetting defects. Precision rates reached 96.28% % and 97.22%, respectively, highlighting the model’s accuracy and generalizability.