Scientific information extraction (SciIE) aims to transform complex scientific literature into structured knowledge, serving as a foundation for scientific understanding and automated discovery. Recent advances in large language models bring new opportunities to SciIE, but the largest and most capable models are prohibitively expensive to fine-tune. We propose ProxyIE, a lightweight decoding-time framework inspired by proxy tuning, tailored to structured extraction. Instead of tuning the large base model, ProxyIE adjusts its output distribution during generation by leveraging token-level logit differences between a smaller fine-tuned expert and its untuned counterpart. Experiments on three SciIE benchmarks show that ProxyIE effectively guides 13B and 70B LLaMA2 models, recovering on average over 90% of the performance gains of fully fine-tuned models. Further analysis reveals that ProxyIE consistently intervenes at structurally critical positions, confirming that its improvements are grounded in meaningful structural guidance. These results highlight ProxyIE as a scalable and practical approach for adapting large language models to SciIE under resource-constrained conditions.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ProxyIE: Parameter-Free Adaptation for Scientific Information Extraction via Proxy Tuning

  • Yang Li,
  • Yajiao Wang,
  • Zhixiong Zhang,
  • Mengting Zhang,
  • Meng Wang,
  • Xin Lin

摘要

Scientific information extraction (SciIE) aims to transform complex scientific literature into structured knowledge, serving as a foundation for scientific understanding and automated discovery. Recent advances in large language models bring new opportunities to SciIE, but the largest and most capable models are prohibitively expensive to fine-tune. We propose ProxyIE, a lightweight decoding-time framework inspired by proxy tuning, tailored to structured extraction. Instead of tuning the large base model, ProxyIE adjusts its output distribution during generation by leveraging token-level logit differences between a smaller fine-tuned expert and its untuned counterpart. Experiments on three SciIE benchmarks show that ProxyIE effectively guides 13B and 70B LLaMA2 models, recovering on average over 90% of the performance gains of fully fine-tuned models. Further analysis reveals that ProxyIE consistently intervenes at structurally critical positions, confirming that its improvements are grounded in meaningful structural guidance. These results highlight ProxyIE as a scalable and practical approach for adapting large language models to SciIE under resource-constrained conditions.