Scholarlink

<p>The increasing complexity and miniaturization of electronic devices have intensified the demand for high-quality printed circuit boards, particularly assembled boards (PCBAs), where early defect detection is essential to prevent severe degradation in device reliability and performance. In this paper, we propose a deep learning-based multi-modal fusion approach with three-mode optical illumination for automated PCBA defect detection. The proposed approach integrates a three-mode optical acquisition system with a purpose-built multi-modal deep learning model to enable robust and efficient defect recognition. In the first stage, we design a custom three-mode optical system integrating coaxial, polarized coaxial and dark-field illumination to enhance shape, edge and surface irregularity cues, thereby generating high-contrast and geometrically stable input images. In the second stage, we develop a multi-modal deep learning framework that performs mid-level feature fusion across three specialized inputs using two complementary strategies: cross-attention-based fusion (IMCAF) and gated Mixture-of-Experts fusion (IMGF). A realistic multi-modal dataset is constructed from defective PCBAs collected in manufacturing environments, comprising 1,774 images and a total of 10,475 annotated defect instances (bounding boxes) across eight defect categories with high morphological similarity and small-scale variations. Experimental results indicate that, under three-mode inputs, the proposed IMGF-based model achieves the highest detection accuracy, while the IMCAF-based model yields slightly lower accuracy with marginally improved inference efficiency. Across backbone comparisons, ConvNeXtV2-T exhibits the most favorable accuracy–efficiency trade-off compared with ResNeSt and Swin Transformer. Relative to single-mode, dual-mode and early-fusion approaches (IECAF), the proposed method delivers substantial performance gains (from approximately 60–70% to around 90% mAP) while maintaining comparable inference latency. These findings demonstrate that the proposed approach provides a flexible and robust solution for automated PCBA defect detection, effectively adapting to modern dense board designs and previously unseen defect patterns in industrial manufacturing environments.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Deep learning-based multi-modal fusion with three-mode optical illumination for robust printed circuit board assembly defect detection

  • 期刊论文

  • 发表日期：

    2026年5月3日

查看全文
  • Sy Hieu Dau,
  • Thi Phuc Dang,
  • Anh Tuan Nguyen

摘要

The increasing complexity and miniaturization of electronic devices have intensified the demand for high-quality printed circuit boards, particularly assembled boards (PCBAs), where early defect detection is essential to prevent severe degradation in device reliability and performance. In this paper, we propose a deep learning-based multi-modal fusion approach with three-mode optical illumination for automated PCBA defect detection. The proposed approach integrates a three-mode optical acquisition system with a purpose-built multi-modal deep learning model to enable robust and efficient defect recognition. In the first stage, we design a custom three-mode optical system integrating coaxial, polarized coaxial and dark-field illumination to enhance shape, edge and surface irregularity cues, thereby generating high-contrast and geometrically stable input images. In the second stage, we develop a multi-modal deep learning framework that performs mid-level feature fusion across three specialized inputs using two complementary strategies: cross-attention-based fusion (IMCAF) and gated Mixture-of-Experts fusion (IMGF). A realistic multi-modal dataset is constructed from defective PCBAs collected in manufacturing environments, comprising 1,774 images and a total of 10,475 annotated defect instances (bounding boxes) across eight defect categories with high morphological similarity and small-scale variations. Experimental results indicate that, under three-mode inputs, the proposed IMGF-based model achieves the highest detection accuracy, while the IMCAF-based model yields slightly lower accuracy with marginally improved inference efficiency. Across backbone comparisons, ConvNeXtV2-T exhibits the most favorable accuracy–efficiency trade-off compared with ResNeSt and Swin Transformer. Relative to single-mode, dual-mode and early-fusion approaches (IECAF), the proposed method delivers substantial performance gains (from approximately 60–70% to around 90% mAP) while maintaining comparable inference latency. These findings demonstrate that the proposed approach provides a flexible and robust solution for automated PCBA defect detection, effectively adapting to modern dense board designs and previously unseen defect patterns in industrial manufacturing environments.