Adapting Pretrained Large-Scale Vision Models for Face Forgery Detection

In the evolving digital realm, generative networks have catalyzed an upsurge in deceptive media, encompassing manipulated facial imagery to tampered text, threatening both personal security and societal stability. While specialized detection networks exist for specific forgery types, their limitations in handling diverse online forgeries and resource constraints necessitate a more holistic approach. This paper presents a pioneering effort to efficiently adapt pre-trained large vision models (LVMs) for the critical task of forgery detection, emphasizing face forgery. Recognizing the inherent challenges in bridging pre-training tasks with forgery detection, we introduce a novel parameter-efficient adaptation strategy. Our investigations highlight the imperative of focusing on detailed, local features to discern forgery indicators. Departing from conventional methods, we propose the Detail-Enhancement Adapter (DE-Adapter), inspired by ‘Unsharp Masking’. By leveraging Gaussian convolution kernels and differential operations, the DE-Adapter enhances detailed representations. With our method, we achieved state-of-the-art performance with only 0.3% network adjustment. Especially when the number of training samples is limited, our method far surpasses other methods. Our work also provides a new perspective for the Uni-Vision Large Model, and we call on more fields to design suitable adapting schemes to expand the capabilities of large models instead of redesigning networks from scratch.