Scholarlink

<p>Large language models (LLMs) excel in information extraction (IE). However, word-level spurious correlations have been observed in prediction errors. Assessing LLMs’ robustness against such risks is crucial for building reliable IE systems. Yet, spurious correlations within LLMs’ semantics remain unevaluated, as existing studies detect them by relying on co-occurrence statistics from training data, which are unavailable for LLMs. To address this challenge, we propose a novel module <b>LLM Causal Discovery</b> (<b>LCD</b>), which leverages statistics encoded in model parameters to identify spurious correlations, grounded in causal discovery. Building on LCD, we introduce a framework <b>Spurious Correlation Evaluator</b> (<b>SCE</b>) to assess robustness using noisy data containing the identified spurious correlations. Our findings, evaluated on state-of-the-art (SOTA) LLMs, reveal their notable fragility and heavy reliance on statistical features. SCE’s perturbation largely outperforms recent robustness evaluation strategies, establishing an SOTA attack system. Additionally, evaluation results can serve as effective feedback to enhance robustness. In summary, SCE tackles the challenge of quantifying word-level spurious correlations in LLMs, providing support for risk control. As a causality-based framework, it also evidences model stability and interpretability. Code and data are available at: <a href="https://github.com/NLPWM-WHU/SCE">https://github.com/NLPWM-WHU/SCE</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

How Robust are Large Language Models Against Word-Level Spurious Correlations? A Causal Discovery Approach

  • 期刊论文

  • 发表日期：

    2026年2月9日

查看全文
  • Xin Miao,
  • Yongqi Li,
  • Hankun Kang,
  • Mayi Xu,
  • Jintao Wen,
  • Yuyang Ren,
  • Tieyun Qian

摘要

Large language models (LLMs) excel in information extraction (IE). However, word-level spurious correlations have been observed in prediction errors. Assessing LLMs’ robustness against such risks is crucial for building reliable IE systems. Yet, spurious correlations within LLMs’ semantics remain unevaluated, as existing studies detect them by relying on co-occurrence statistics from training data, which are unavailable for LLMs. To address this challenge, we propose a novel module LLM Causal Discovery (LCD), which leverages statistics encoded in model parameters to identify spurious correlations, grounded in causal discovery. Building on LCD, we introduce a framework Spurious Correlation Evaluator (SCE) to assess robustness using noisy data containing the identified spurious correlations. Our findings, evaluated on state-of-the-art (SOTA) LLMs, reveal their notable fragility and heavy reliance on statistical features. SCE’s perturbation largely outperforms recent robustness evaluation strategies, establishing an SOTA attack system. Additionally, evaluation results can serve as effective feedback to enhance robustness. In summary, SCE tackles the challenge of quantifying word-level spurious correlations in LLMs, providing support for risk control. As a causality-based framework, it also evidences model stability and interpretability. Code and data are available at: https://github.com/NLPWM-WHU/SCE.