Scholarlink

<p>In recent years, deepfake techniques represented by Generative Adversarial Networks (GANs) have achieved the capability to generate highly realistic images and video content. This poses a serious threat to personal privacy, social discourse security, and even national security. However, most existing approaches focus heavily on network architecture design while neglecting the semantic understanding capabilities of existing models, resulting in insufficient generalization performance in complex scenarios. To address these challenges, this paper proposes a deepfake detection framework based on multi-level fine-tuning strategies. The method constructs an Encoder-Decoder architecture detection system. The encoder employs a fully parameter fine-tuned fusion model of Vision Transformer (ViT) and Visual Geometry Group Network (VGGNet) to extract global and local image features, respectively. The decoder incorporates a large language model fine-tuned via LoRA to further fuse multi-scale features and perform semantic reasoning and classification. Experiments conducted on public datasets such as FaceForensics++ and OpenForensics demonstrate that the proposed method achieves 95.7% accuracy, 99.0% validation accuracy, 96.1% recall, and an F1 score of 97.2% on the DeepFake dataset, significantly outperforming mainstream baseline models including CNN, ResNet, and Swin Transformer.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Finetune is all you need: DeepFake detection via LLM and neural network fine-tuning

  • 期刊论文

  • 发表日期：

    2026年4月25日

查看全文
  • Longhao Wang,
  • Ruibin Yan,
  • Manchun Cai,
  • Xiaoran Xie,
  • Yuzheng Tian

摘要

In recent years, deepfake techniques represented by Generative Adversarial Networks (GANs) have achieved the capability to generate highly realistic images and video content. This poses a serious threat to personal privacy, social discourse security, and even national security. However, most existing approaches focus heavily on network architecture design while neglecting the semantic understanding capabilities of existing models, resulting in insufficient generalization performance in complex scenarios. To address these challenges, this paper proposes a deepfake detection framework based on multi-level fine-tuning strategies. The method constructs an Encoder-Decoder architecture detection system. The encoder employs a fully parameter fine-tuned fusion model of Vision Transformer (ViT) and Visual Geometry Group Network (VGGNet) to extract global and local image features, respectively. The decoder incorporates a large language model fine-tuned via LoRA to further fuse multi-scale features and perform semantic reasoning and classification. Experiments conducted on public datasets such as FaceForensics++ and OpenForensics demonstrate that the proposed method achieves 95.7% accuracy, 99.0% validation accuracy, 96.1% recall, and an F1 score of 97.2% on the DeepFake dataset, significantly outperforming mainstream baseline models including CNN, ResNet, and Swin Transformer.