CFDT-CLIP:fabric anomaly detection with collaborative fusion under dual-text prompts
摘要
Defect detection in complex textiles is a key challenge. Contrastive language-image pre-training (CLIP) performs well but is restricted by its single-text prompt from simultaneous segmentation and classification. A textile defect detection method based on collaborative fusion of dual-text prompts for CLIP (CFDT-CLIP) is proposed to integrate pixel-level and image-level defect detection effectively. A pixel-prioritized module (PPM) focuses on pixel-level segmentation, with a text data augmentation (TDA) module enhancing the diversity of feature texts containing defect information. An image-priority module (IPM) assigns a learnable generic text prompt to each image (ignoring specific defect categories) to differentiate defective/good images and prioritize image-level classification. Via the mixed output module (MOM), a learnable collaborative fusion matrix (LCFM) integrates and normalizes anomaly scores/maps from PPM and IPM for final results. Tested on textile datasets, the algorithm outperforms mainstream CLIP models in both tasks. The dual-branch PPM/IPM provides technical underpinnings for shortening inference time, enabling high-performance parallel computing, and empowering supercomputing.