Scholarlink

<p>Retrieval-Augmented Generation (RAG) has shown remarkable results to address the hallucinations of Large Language Models (LLMs), which usually uses a large external corpus to supplement knowledge. However, with the development of LLMs, the internal knowledge of LLMs has expanded significantly, thus causing significant knowledge redundancy between the external corpus and LLMs. On the one hand, the indexing cost of dense retrieval is highly related to the corpus size and thus significant redundant knowledge intensifies the dense retrieval’s workload. On the other hand, the redundant knowledge in the external corpus is not helpful to LLMs and our exploratory analysis shows that it instead hurts the RAG performance on those questions which the LLM can answer by itself. To address these issues, <b>Zero-RAG</b> is proposed to tackle these challenges. Specifically, a <b>Mastery-Score</b> metric is introduced to identify and prune redundant knowledge in the RAG corpus. After pruning, answers to “mastered” questions rely primarily on internal knowledge of the LLM. To better exploit this capacity, a <i>Query Router</i> and a <i>Noise-Tolerant Tuning</i> strategy are employed, thereby mitigating the distraction of irrelevant documents and enhancing the utilisation of internal knowledge under the pruned corpus. Experimental results show that Zero-RAG prunes the Wikipedia corpus by 30% and accelerates the retrieval stage by 27%, without compromising RAG’s performance (average degradation &lt;2 points).</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Zero-RAG: towards retrieval-augmented generation with zero redundant knowledge

  • 期刊论文

  • 发表日期：

    2026年8月5日

查看全文
  • Qi Luo,
  • Xiaonan Li,
  • Junqi Dai,
  • Shuang Chen,
  • Yining Zheng,
  • Xipeng Qiu

摘要

Retrieval-Augmented Generation (RAG) has shown remarkable results to address the hallucinations of Large Language Models (LLMs), which usually uses a large external corpus to supplement knowledge. However, with the development of LLMs, the internal knowledge of LLMs has expanded significantly, thus causing significant knowledge redundancy between the external corpus and LLMs. On the one hand, the indexing cost of dense retrieval is highly related to the corpus size and thus significant redundant knowledge intensifies the dense retrieval’s workload. On the other hand, the redundant knowledge in the external corpus is not helpful to LLMs and our exploratory analysis shows that it instead hurts the RAG performance on those questions which the LLM can answer by itself. To address these issues, Zero-RAG is proposed to tackle these challenges. Specifically, a Mastery-Score metric is introduced to identify and prune redundant knowledge in the RAG corpus. After pruning, answers to “mastered” questions rely primarily on internal knowledge of the LLM. To better exploit this capacity, a Query Router and a Noise-Tolerant Tuning strategy are employed, thereby mitigating the distraction of irrelevant documents and enhancing the utilisation of internal knowledge under the pruned corpus. Experimental results show that Zero-RAG prunes the Wikipedia corpus by 30% and accelerates the retrieval stage by 27%, without compromising RAG’s performance (average degradation <2 points).