Scholarlink

<p>Large Language Models (LLMs) have transformed natural language processing, yet their static knowledge bases present significant challenges, such as temporal obsolescence and factual inconsistencies, limiting their effectiveness in real-world applications. Knowledge editing offers an efficient alternative to full-model retraining by enabling targeted updates to an LLM’s knowledge. However, existing approaches often overlook the ripple effects of edits on interconnected information, resulting in limited generalization. To address this, we propose a new model that leverages g<Emphasis Type="Underline">R</Emphasis>aph tr<Emphasis Type="Underline">A</Emphasis>nsformers for <Emphasis Type="Underline">K</Emphasis>nowledge <Emphasis Type="Underline">E</Emphasis>diting in large <Emphasis Type="Underline">L</Emphasis>anguage models (RAKEL). RAKEL employs nucleus sampling to construct subgraphs that capture the cascading impacts of knowledge edits, preserving contextual relationships. A graph transformer then encodes these subgraphs, modeling both local and global dependencies within the knowledge graph to improve the representation of interrelated knowledge. Additionally, a constrained learning strategy ensures alignment between the edited model’s output distribution and that of the original model, reducing overfitting to updated knowledge. We evaluate RAKEL on two datasets, where it outperforms state-of-the-art baselines across multiple metrics. These results highlight RAKEL’s ability to deliver precise and generalizable knowledge updates, particularly in complex multi-hop reasoning scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leveraging Graph Transformers for Knowledge Editing in Large Language Models

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月5日

查看全文
  • Wang Gao,
  • Xintong Li,
  • Yize Cao,
  • Gexia Zhang,
  • Wuyang Yang,
  • Gang Hu

摘要

Large Language Models (LLMs) have transformed natural language processing, yet their static knowledge bases present significant challenges, such as temporal obsolescence and factual inconsistencies, limiting their effectiveness in real-world applications. Knowledge editing offers an efficient alternative to full-model retraining by enabling targeted updates to an LLM’s knowledge. However, existing approaches often overlook the ripple effects of edits on interconnected information, resulting in limited generalization. To address this, we propose a new model that leverages gRaph trAnsformers for Knowledge Editing in large Language models (RAKEL). RAKEL employs nucleus sampling to construct subgraphs that capture the cascading impacts of knowledge edits, preserving contextual relationships. A graph transformer then encodes these subgraphs, modeling both local and global dependencies within the knowledge graph to improve the representation of interrelated knowledge. Additionally, a constrained learning strategy ensures alignment between the edited model’s output distribution and that of the original model, reducing overfitting to updated knowledge. We evaluate RAKEL on two datasets, where it outperforms state-of-the-art baselines across multiple metrics. These results highlight RAKEL’s ability to deliver precise and generalizable knowledge updates, particularly in complex multi-hop reasoning scenarios.