Scholarlink

<p>Large language models (LLMs) have shown promise for scientific data extraction from publications but rely on manual prompt refinement. We present an expert-grounded automatic prompt optimization framework that enhances LLM entity extraction reliability. Using high-entropy alloy lattice constant extraction as a testbed, we optimized prompts for Claude 3.5 Sonnet through feedback cycles on seven expert-annotated publications. Despite a modest optimization budget, recall improved from 0.27 to &gt;&#xa0;0.9, demonstrating that a small, expert-annotated dataset can yield significant improvements. The optimized prompt was applied to extract lattice constants from 2267 publications, yielding data for 1861 compositions. The optimized prompt transferred effectively to newer models: Claude 4.5 Sonnet, GPT-5, and Gemini 2.5 Flash. Analysis revealed three categories of LLM failure modes: contextual hallucination, semantic misinterpretation, and unit conversion errors, emphasizing the need for validation protocols. These results establish feedback-guided prompt optimization as a low-cost, transferable methodology for reliable scientific data extraction, providing a scalable pathway for complex LLM-assisted research tasks.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Expert-Grounded Automatic Prompt Engineering for Extracting Lattice Constants of High-Entropy Alloys from Scientific Publications Using Large Language Models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月15日

查看全文
  • Shunshun Liu,
  • Talon R. Booth,
  • Yangfeng Ji,
  • Wesley Reinhart,
  • Prasanna V. Balachandran

摘要

Large language models (LLMs) have shown promise for scientific data extraction from publications but rely on manual prompt refinement. We present an expert-grounded automatic prompt optimization framework that enhances LLM entity extraction reliability. Using high-entropy alloy lattice constant extraction as a testbed, we optimized prompts for Claude 3.5 Sonnet through feedback cycles on seven expert-annotated publications. Despite a modest optimization budget, recall improved from 0.27 to > 0.9, demonstrating that a small, expert-annotated dataset can yield significant improvements. The optimized prompt was applied to extract lattice constants from 2267 publications, yielding data for 1861 compositions. The optimized prompt transferred effectively to newer models: Claude 4.5 Sonnet, GPT-5, and Gemini 2.5 Flash. Analysis revealed three categories of LLM failure modes: contextual hallucination, semantic misinterpretation, and unit conversion errors, emphasizing the need for validation protocols. These results establish feedback-guided prompt optimization as a low-cost, transferable methodology for reliable scientific data extraction, providing a scalable pathway for complex LLM-assisted research tasks.