Reference image guided industrial defect generation with specified region and strength

Collecting and labeling a large-scale defect dataset to build a high-performance defect inspection model is a challenge. Diffusion models show great potential for various image synthesis tasks, offering a solution to this problem. Diffusion-based methods usually use text prompts as guidance. However, industrial images are often monotonous in content and semantically ambiguous, making it difficult to describe defect details in natural language. Otherwise, for industrial defect inspection, the shape, position, and diversity of the defect are crucial. To address the above challenges, we propose a Reference Image Guided Industrial Defect Generation with specified region and strength method (RIG-IDG). Firstly, we use the reference defect image to provide more precise and detailed information about the defect characteristics, and introduce a style loss to supervise the model in generating an image similar to the reference. Secondly, we use the mask to specify the region of the generated defect and introduce a strength control and background preservation mechanism in the sampling stage to balance diversity and consistency in the generated samples. We conduct extensive experiments on various datasets, demonstrating that the proposed method outperforms other GAN- and diffusion-based defect generation methods, achieving state-of-the-art performance in generating high-quality defect images and enhancing the defect inspection dataset. Quantitative experiments also confirm that our method can generate high-quality, diverse, and visually realistic defect images, allowing for control over the locations and strengths of the generated defects, and the generated samples align well with the style of the reference defect.