Scholarlink

<p>Large language models (LLMs) are becoming essential tools for scientometric analyses, including literature-based research assessment, knowledge structure analysis, and citation context interpretation. However, when the evidentiary basis and contextual premises underlying an LLM’s output are unclear, its results cannot be reliably used for scientometric interpretation. This study therefore addresses evidence access design as a foundational requirement for transparent and stable LLM-based scientometric pipelines. In scientific literature, interpretive premises such as evaluation criteria, scope of applicability, and experimental conditions are often distributed across multiple sections of a paper. While embedding-based paragraph retrieval effectively captures local semantic similarity, it has structural limitations in assembling premise-defining evidence into a coherent interpretive configuration. To address this, we propose B<span>-page</span>, a graph-based evidence augmentation framework that preserves embedding-based retrieval as a baseline while using intra-document structural connectivity as a constrained expansion mechanism. Rather than replacing retrieval, B<span>-page</span> reconfigures the evidentiary structure within which research claims are interpreted. Using QASPER as a controlled full-paper testbed with paragraph-level gold evidence, we evaluate evidence access independently of LLM generation. Results indicate that graph-based augmentation can improve recall under constrained budgets in specific conditions, while excessive expansion introduces structural noise. More importantly, our analysis demonstrates that evidence-access design systematically shapes how interpretive premises are assembled under fixed budgets. By formalizing these design principles, this work contributes to the DESIGN and TOOLS perspectives of the LLMs for Scientometrics (LLM4SCIM) special issue.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evidence access design for llm-based scientometrics: graph-based evidence augmentation over semantic retrieval

  • 期刊论文

  • 发表日期：

    2026年7月8日

查看全文
  • Yunseop Lee,
  • Minseok Hur,
  • Moohong Min

摘要

Large language models (LLMs) are becoming essential tools for scientometric analyses, including literature-based research assessment, knowledge structure analysis, and citation context interpretation. However, when the evidentiary basis and contextual premises underlying an LLM’s output are unclear, its results cannot be reliably used for scientometric interpretation. This study therefore addresses evidence access design as a foundational requirement for transparent and stable LLM-based scientometric pipelines. In scientific literature, interpretive premises such as evaluation criteria, scope of applicability, and experimental conditions are often distributed across multiple sections of a paper. While embedding-based paragraph retrieval effectively captures local semantic similarity, it has structural limitations in assembling premise-defining evidence into a coherent interpretive configuration. To address this, we propose B-page, a graph-based evidence augmentation framework that preserves embedding-based retrieval as a baseline while using intra-document structural connectivity as a constrained expansion mechanism. Rather than replacing retrieval, B-page reconfigures the evidentiary structure within which research claims are interpreted. Using QASPER as a controlled full-paper testbed with paragraph-level gold evidence, we evaluate evidence access independently of LLM generation. Results indicate that graph-based augmentation can improve recall under constrained budgets in specific conditions, while excessive expansion introduces structural noise. More importantly, our analysis demonstrates that evidence-access design systematically shapes how interpretive premises are assembled under fixed budgets. By formalizing these design principles, this work contributes to the DESIGN and TOOLS perspectives of the LLMs for Scientometrics (LLM4SCIM) special issue.