The rapid advancements in sophisticated image generation technologies and the proliferation of novel generative models pose significant challenges to social trust and information security. In this case, developing detectors capable of generalizing to unseen generative models is of significance. However, due to the inherent Out-of-Distribution (OOD) nature, existing detection methods tend to exhibit poor performance when handling images derived from new, unseen models. To address this issue, we propose MambaClip-DeFuseNet (MCDF), a novel generative images detection framework, with a CLIP-Mamba backbone for its superior OOD generalization capabilities over traditional vision transformers. This backbone choice delivers a shape bias that is more consistent with the human visual perception, and it has a heightened sensitivity to high-frequency artifacts. Specifically, an Einstein FFT Disentangled Block (EFDB) is embedded in MCDF to refine deep image features by disentangling subtle spectral traces from the semantic content. During the training process, we design a hybrid spectro-contrastive loss that provides classification supervision with a spectral contrastive regularizer for guiding the salient image features extraction. Extensive experiments on 15 diverse generative models demonstrate the superiority of our MCDF, especially for challenging models like DALLE-2 and Midjourney. Maintaining a parameter-efficient design, our framework sets a new baseline for CLIP-Mamba-based image forensics and reveals the potential of spectral disentanglement in this field. Our code can be found at Here .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MambaGuard: A CLIP-Mamba Approach for OOD Generated Image Detection

  • Xinchang Wang,
  • Yuechen Zhang,
  • Zihao Guo,
  • Wenyao Qiu,
  • Chunyang Cheng,
  • Hui Li

摘要

The rapid advancements in sophisticated image generation technologies and the proliferation of novel generative models pose significant challenges to social trust and information security. In this case, developing detectors capable of generalizing to unseen generative models is of significance. However, due to the inherent Out-of-Distribution (OOD) nature, existing detection methods tend to exhibit poor performance when handling images derived from new, unseen models. To address this issue, we propose MambaClip-DeFuseNet (MCDF), a novel generative images detection framework, with a CLIP-Mamba backbone for its superior OOD generalization capabilities over traditional vision transformers. This backbone choice delivers a shape bias that is more consistent with the human visual perception, and it has a heightened sensitivity to high-frequency artifacts. Specifically, an Einstein FFT Disentangled Block (EFDB) is embedded in MCDF to refine deep image features by disentangling subtle spectral traces from the semantic content. During the training process, we design a hybrid spectro-contrastive loss that provides classification supervision with a spectral contrastive regularizer for guiding the salient image features extraction. Extensive experiments on 15 diverse generative models demonstrate the superiority of our MCDF, especially for challenging models like DALLE-2 and Midjourney. Maintaining a parameter-efficient design, our framework sets a new baseline for CLIP-Mamba-based image forensics and reveals the potential of spectral disentanglement in this field. Our code can be found at Here .