Scholarlink

<p>Large Language Models (LLMs) have shown strong performance on a wide range of natural language processing tasks, yet they still struggle with multi-hop question answering, which demands complex reasoning over multiple evidence sources. While extending context windows has been proposed as a solution, this approach remains cost-prohibitive and suffers from attention degradation over long contexts. Retrieval-Augmented Generation (RAG) offers a more efficient alternative by selectively incorporating external information. In this paper, we present a systematic evaluation of multiple RAG configurations using lexical (BM25) and semantic retrievers (msmarco-bert-base-dot-v, ColBERTv2), as well as graph-based approaches (HippoRAG). We assess performance across four QA datasets (HotpotQA, 2Wiki, MuSiQue, and LoCoMo) employing accessible LLMs such as GPT-4o-mini, Qwen2.5-14B-Instruct, and o3-mini. Our experiments analyze the trade-offs between retrieval depth, cost, and model performance, showing that medium-scale LLMs can achieve competitive results through retrieval scaling. However, for highly compositional tasks like MuSiQue, larger reasoning models remain advantageous. Our results offer practical insights into optimizing RAG pipelines in budget-constrained environments, and we provide a publicly available framework to facilitate further research.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Analyzing Retrieval Scaling in RAG Systems for Complex QA Benchmarks

  • 期刊论文

  • 发表日期：

    2025年8月22日

查看全文
  • Jhon Stewar Rayo Mosquera,
  • Rubén Francisco Manrique

摘要

Large Language Models (LLMs) have shown strong performance on a wide range of natural language processing tasks, yet they still struggle with multi-hop question answering, which demands complex reasoning over multiple evidence sources. While extending context windows has been proposed as a solution, this approach remains cost-prohibitive and suffers from attention degradation over long contexts. Retrieval-Augmented Generation (RAG) offers a more efficient alternative by selectively incorporating external information. In this paper, we present a systematic evaluation of multiple RAG configurations using lexical (BM25) and semantic retrievers (msmarco-bert-base-dot-v, ColBERTv2), as well as graph-based approaches (HippoRAG). We assess performance across four QA datasets (HotpotQA, 2Wiki, MuSiQue, and LoCoMo) employing accessible LLMs such as GPT-4o-mini, Qwen2.5-14B-Instruct, and o3-mini. Our experiments analyze the trade-offs between retrieval depth, cost, and model performance, showing that medium-scale LLMs can achieve competitive results through retrieval scaling. However, for highly compositional tasks like MuSiQue, larger reasoning models remain advantageous. Our results offer practical insights into optimizing RAG pipelines in budget-constrained environments, and we provide a publicly available framework to facilitate further research.