This chapter presents an end-to-end pipeline for generating, refining, and evaluating adversarial patches to compromise facial biometric systems for forensic analysis and security testing. We used FGSM to generate adversarial noise targeting our identity-detection classifier. We employed a diffusion model to generate Impersonation patch attacks, with additional Gaussian smoothing and adaptive brightness correction during synthetic adversarial patch generation. The refined patch is applied to facial images to test its ability to evade recognition systems while maintaining natural visual characteristics. A vision transformer (ViT)-GPT2 model generates captions that provide a semantic description of a person’s identity for Adv images, supporting forensic interpretation and documentation for identity-evasion attacks and recognition. The pipeline evaluates changes in identity classification, captioning results, and the vulnerability of facial identity verification and expression to adversarial attacks. Therefore, detecting and mitigating attacks from these adversaries is necessary in forensic settings using perceptual hashing. We successfully detected and localized generated adversaries, achieving an SSIM of 0.95%.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Diffusion-Driven Deceptive Patches: Adversarial Manipulation and Forensic Detection in Facial Identity Verification

  • Shahrzad Sayyafzadeh,
  • Hongmei Chi,
  • Shonda Bernadin

摘要

This chapter presents an end-to-end pipeline for generating, refining, and evaluating adversarial patches to compromise facial biometric systems for forensic analysis and security testing. We used FGSM to generate adversarial noise targeting our identity-detection classifier. We employed a diffusion model to generate Impersonation patch attacks, with additional Gaussian smoothing and adaptive brightness correction during synthetic adversarial patch generation. The refined patch is applied to facial images to test its ability to evade recognition systems while maintaining natural visual characteristics. A vision transformer (ViT)-GPT2 model generates captions that provide a semantic description of a person’s identity for Adv images, supporting forensic interpretation and documentation for identity-evasion attacks and recognition. The pipeline evaluates changes in identity classification, captioning results, and the vulnerability of facial identity verification and expression to adversarial attacks. Therefore, detecting and mitigating attacks from these adversaries is necessary in forensic settings using perceptual hashing. We successfully detected and localized generated adversaries, achieving an SSIM of 0.95%.