Synthetic Image Detection Using Mixture of Knowledge Distillation from Vision-Language Models
摘要
Within the domain of advanced image authentication, the rapid advancement of generative models, such as diffusion models and Generative Adversarial Networks (GANs), poses novel challenges in discriminating between real and synthetic images. In response to this challenge, our present paper introduces an innovative approach that leverages the transfer of knowledge from Vision-Language Models (VLMs), such as BLIP2 and ViTGPT2, to a streamlined Convolutional Neural Network (CNN)-based architecture, specifically the ResNet18 architecture. Integrating advanced learning methodologies with a compact network design significantly advances synthetic image detection. We employ a knowledge distillation framework, enabling ResNet18 to assimilate and incorporate rich feature representations from two VLMs. This methodology enhances the model’s ability to differentiate real images from synthetic ones and alleviates computational burdens. Our experimental results demonstrate the model’s efficiency and precision in synthetic image detection, achieving a high average accuracy of 99.48%. The fusion of VLMs insights into ResNet18, facilitated by a crucial knowledge transfer process, embodies a notable advancement in the field, establishing a new benchmark for image authentication in the era of advanced synthetic image detection. To support reproducible research, the source code of the proposed model will be released at: https://github.com/HessenUPHF/Synthetic-Image-Detection-using-Mixture-of-Knowledge-Distillation-from-Vision-Language-Models.git