Few-shot mask-guided controllable anomaly image generation via inpainting diffusion for surface defect detection
摘要
Data-driven surface defect detection in intelligent manufacturing heavily relies on well-annotated defect samples, yet such data are often scarce in real production lines. Synthesizing defect samples with image generation models provides an effective way to alleviate this limitation. However, existing methods under limited data often struggle to generate defects that blend naturally with surrounding backgrounds, while also offering limited controllability and diversity. To address these issues, we propose MaCoDiff, a mask-guided controllable defect image generation framework based on an inpainting diffusion model, which can synthesize high-quality defect images with pixel-level annotations from only a few defect samples. We introduce a semantic-aware dual-branch loss with two text-conditioned objectives. One branch focuses on fine-grained defect appearance modeling, while the other explicitly constrains defect-background semantic consistency, enabling the generated defects to better preserve both realism and contextual coherence. We further design a saliency control module (SCM) to modulate defect-related semantic contributions in the latent space, allowing flexible adjustment of defect saliency to simulate different severity levels. In addition, a spatial transformation-based mask generation strategy is employed to provide mask conditions with diverse shapes and locations, further improving the diversity of generated defects. Extensive experiments demonstrate that the proposed method can effectively control the location, morphology, and saliency of generated defects, producing realistic defect images that align well with the target masks. Compared with state-of-the-art methods, the generated image-mask pairs more effectively improve the performance of defect segmentation models, providing a practical data augmentation solution for few-shot surface visual inspection.