Diffusion-Driven Deceptive Patches: Adversarial Manipulation and Forensic Detection in Facial Identity Verification
摘要
This chapter presents an end-to-end pipeline for generating, refining, and evaluating adversarial patches to compromise facial biometric systems for forensic analysis and security testing. We used FGSM to generate adversarial noise targeting our identity-detection classifier. We employed a diffusion model to generate Impersonation patch attacks, with additional Gaussian smoothing and adaptive brightness correction during synthetic adversarial patch generation. The refined patch is applied to facial images to test its ability to evade recognition systems while maintaining natural visual characteristics. A vision transformer (ViT)-GPT2 model generates captions that provide a semantic description of a person’s identity for Adv images, supporting forensic interpretation and documentation for identity-evasion attacks and recognition. The pipeline evaluates changes in identity classification, captioning results, and the vulnerability of facial identity verification and expression to adversarial attacks. Therefore, detecting and mitigating attacks from these adversaries is necessary in forensic settings using perceptual hashing. We successfully detected and localized generated adversaries, achieving an SSIM of 0.95%.