Scholarlink

<p>Recent advances in deep learning research have shown remarkable achievements across many tasks in computer vision (CV) and natural language processing (NLP). At the intersection of CV and NLP is the problem of image captioning, where the related models’ robustness against adversarial attacks has not been well studied. This paper presents a novel adversarial attack strategy, attention-based image captioning attack (AICAttack), designed to attack image captioning models through subtle perturbations to images. Operating within a black-box attack scenario, our algorithm requires no access to the target model’s architecture, parameters, or gradient information. We introduce an attention-based candidate selection mechanism that identifies the optimal pixels for attack, followed by a customized differential evolution method to optimize the perturbations of the pixels’ RGB values. We demonstrate AICAttack’s effectiveness through extensive experiments on benchmark datasets against multiple victim models. The experimental results demonstrate that our method outperforms current leading-edge techniques by achieving consistently higher attack success rates.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

AICAttack: Adversarial Image Captioning Attack with Attention-based Optimization

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年7月7日

查看全文
  • Jiyao Li,
  • Mingze Ni,
  • Yifei Dong,
  • Tianqing Zhu,
  • Yongshun Gong,
  • Wei Liu

摘要

Recent advances in deep learning research have shown remarkable achievements across many tasks in computer vision (CV) and natural language processing (NLP). At the intersection of CV and NLP is the problem of image captioning, where the related models’ robustness against adversarial attacks has not been well studied. This paper presents a novel adversarial attack strategy, attention-based image captioning attack (AICAttack), designed to attack image captioning models through subtle perturbations to images. Operating within a black-box attack scenario, our algorithm requires no access to the target model’s architecture, parameters, or gradient information. We introduce an attention-based candidate selection mechanism that identifies the optimal pixels for attack, followed by a customized differential evolution method to optimize the perturbations of the pixels’ RGB values. We demonstrate AICAttack’s effectiveness through extensive experiments on benchmark datasets against multiple victim models. The experimental results demonstrate that our method outperforms current leading-edge techniques by achieving consistently higher attack success rates.