Retrieval-augmented generation (RAG) is a technique that generates responses to questions by leveraging information retrieved from databases and large language models (LLM). However, with the advancements of LLMs and RAG, there are growing concerns about the potential for unintended responses resulting from malicious attacks, such as prompt injection attacks. To use LLMs and RAG techniques more safely, it is important to understand the vulnerabilities inherent in RAG by examining various attack methods. This paper proposes an untargeted adversarial input attack method specifically aimed at RAG. The proposed method involves adding adversarial strings to the prompts that are used as queries for RAG. By optimizing the adversarial strings to minimize the similarity between the prompt with adversarial strings and the relevant information contained in RAG’s database during information retrieval, we can effectively reduce the accuracy of the generated responses. Additionally, by applying a poisoning attack that injects sentences with adversarial strings into RAG’s database, we further decrease the accuracy of the responses. Through evaluation experiments, we confirmed that the proposed adversarial input attack and the poisoning attack were successful across multiple models and datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluation of Adversarial Input Attacks in Retrieval-Augmented Generation Using Large Language Models

  • Kento Hasegawa,
  • Seira Hidano,
  • Kazuhide Fukushima

摘要

Retrieval-augmented generation (RAG) is a technique that generates responses to questions by leveraging information retrieved from databases and large language models (LLM). However, with the advancements of LLMs and RAG, there are growing concerns about the potential for unintended responses resulting from malicious attacks, such as prompt injection attacks. To use LLMs and RAG techniques more safely, it is important to understand the vulnerabilities inherent in RAG by examining various attack methods. This paper proposes an untargeted adversarial input attack method specifically aimed at RAG. The proposed method involves adding adversarial strings to the prompts that are used as queries for RAG. By optimizing the adversarial strings to minimize the similarity between the prompt with adversarial strings and the relevant information contained in RAG’s database during information retrieval, we can effectively reduce the accuracy of the generated responses. Additionally, by applying a poisoning attack that injects sentences with adversarial strings into RAG’s database, we further decrease the accuracy of the responses. Through evaluation experiments, we confirmed that the proposed adversarial input attack and the poisoning attack were successful across multiple models and datasets.