Scholarlink

Retrieval-Augmented Generation (RAG) systems enhance large language models by incorporating external knowledge from document corpora. Evaluating these systems requires high-quality, traceable test datasets. While existing frameworks like RAGAS support automated testset synthesis, they omit structural metadata such as context node identifiers, hindering reproducibility and interpretability. This paper addresses these limitations by introducing a context-ID-aware enhancement to RAGAS and developing an interactive visualization tool based on pyvis. Our method embeds unique node identifiers within reference contexts, enabling end-to-end traceability from QA pairs to original knowledge graph nodes. We augment the visualization interface with custom JavaScript and HTML components, allowing users to search and analyze multi-hop reasoning chains. Through case studies on legal and ESG documents, we demonstrate improved dataset transparency, detection of false-positive multi-hop inferences, and robust evaluation workflows. The proposed system transforms static testsets into interactive, auditable resources, advancing RAG evaluation reliability and explainability.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Enhancing Traceability and Interpretability of Datasets for RAG Evaluation: A Context-ID-Aware and Graph-Based Visualization Approach

  • 会议论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年

查看全文
  • Beomseok Kim,
  • Jinhong Yang

摘要

Retrieval-Augmented Generation (RAG) systems enhance large language models by incorporating external knowledge from document corpora. Evaluating these systems requires high-quality, traceable test datasets. While existing frameworks like RAGAS support automated testset synthesis, they omit structural metadata such as context node identifiers, hindering reproducibility and interpretability. This paper addresses these limitations by introducing a context-ID-aware enhancement to RAGAS and developing an interactive visualization tool based on pyvis. Our method embeds unique node identifiers within reference contexts, enabling end-to-end traceability from QA pairs to original knowledge graph nodes. We augment the visualization interface with custom JavaScript and HTML components, allowing users to search and analyze multi-hop reasoning chains. Through case studies on legal and ESG documents, we demonstrate improved dataset transparency, detection of false-positive multi-hop inferences, and robust evaluation workflows. The proposed system transforms static testsets into interactive, auditable resources, advancing RAG evaluation reliability and explainability.