Towards Generalizable and Interpretable AI-Modified Image Detectors

Recent successes in image generation models have led to some social problems because users can generate realistic fake, i.e. AI-modified, images without any obstacles. In contrast, recent fake image detectors still have two limitations. First, they cannot generalize well across image generation strategies, like image inpainting and style transfer. Second, they lack interpretability, which reduces their reliability. To tackle these two issues, we propose an AI-modified image detection method with three designs: 1) for generalizability, we select task-relevant features from a pre-trained network backbone based on the Wasserstein distance; 2) for interpretability, we use an interpretable hierarchical AI-modified image detection process; 3) for representability, we tailor the spatial and channel attention mechanisms for our model to achieve optimal performance. Experiments show that our method outperforms state-of-the-art methods, and all its modules improve performance. The code is available at https://github.com/XinshuangL/GenIntAIDet .