<p>The proliferation of advanced generative models presents a significant threat to the authenticity of digital visual content, creating an urgent demand for robust forgery detection technologies. A primary challenge lies in developing a unified and generalizable framework that can not only classify an image as authentic or forged but also precisely localize the manipulated regions, all while maintaining computational efficiency and adapting to unseen manipulation techniques. To address these challenges, we introduce Aperture, a lightweight and effective framework for joint deepfake detection and localization. Our framework integrates a novel patch-aware classifier that fuses local and global features to enhance classification robustness. For localization, a segmentation module guided by conditional queries is employed to generate fine-grained masks of tampered regions. Crucially, we propose a pseudo-label-guided test-time adaptation mechanism, which leverages the classifier’s patch-level predictions to dynamically supervise the segmentation head. This allows the model to improve its robustness against out-of-distribution data encountered during testing.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Aperture: a patch-aware framework for joint forgery detection and localization

  • Anwen Yang,
  • Jian Zhao,
  • Yuchen Yuan,
  • Tianle Zhang,
  • Yuchu Jiang,
  • Jiaming Chu,
  • Xin Zhang,
  • Xu Yang,
  • Lei Jin,
  • Chi Zhang,
  • Zhaofeng He

摘要

The proliferation of advanced generative models presents a significant threat to the authenticity of digital visual content, creating an urgent demand for robust forgery detection technologies. A primary challenge lies in developing a unified and generalizable framework that can not only classify an image as authentic or forged but also precisely localize the manipulated regions, all while maintaining computational efficiency and adapting to unseen manipulation techniques. To address these challenges, we introduce Aperture, a lightweight and effective framework for joint deepfake detection and localization. Our framework integrates a novel patch-aware classifier that fuses local and global features to enhance classification robustness. For localization, a segmentation module guided by conditional queries is employed to generate fine-grained masks of tampered regions. Crucially, we propose a pseudo-label-guided test-time adaptation mechanism, which leverages the classifier’s patch-level predictions to dynamically supervise the segmentation head. This allows the model to improve its robustness against out-of-distribution data encountered during testing.