Scholarlink

<p>When model-based intelligent agents tackle real-world tasks, they require continuous knowledge updating to align with evolving world knowledge and dynamic user demands. To address this challenge, model editing methods update specific weights of Large Language Models (LLMs) without retraining; however, as edits accumulate, direct weight updates still lead to the catastrophic result of forgetting. In response, retrieval-based editing methods avoid modifying the original parameters directly by editing replicas and retrieving as well as activating the corresponding replicas during inference. Nevertheless, they remain confined to performing updates within a single layer and suffer from the coordination between editing and retrieval activation. To address these issues, we propose the Activation Control Layer-based continual Knowledge Editing (ACKE) method. First, we design a multi-layer replica framework that shares edits across FFN layers, thereby alleviating single-layer overload and reducing parameter merging frequency. Second, we devise an independent activation control layer integrated with an inverse similarity mechanism that decouples activation from editing, enabling more sensitive discrimination between edited and irrelevant knowledge for more robust retrieval. ACKE achieves an average score of 92.8% on ZsRE sequential editing and a Perplexity score of 1.88 on SelfCheckGPT. Compared with existing methods, it demonstrates stronger large-scale continual editing capability, providing a basis for more robust long-term knowledge updating of LLMs. The code is available at <a href="https://github.com/soraue/ACKE">https://github.com/soraue/ACKE</a>.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Activation Trained Layer Based Controls Continual Knowledge Editing Approach

  • 期刊论文

  • 发表日期：

    2026年6月24日

查看全文
  • Yilin Huang,
  • Yimin Wen,
  • Junying Zeng,
  • Xudong Jia,
  • Tao Chen

摘要

When model-based intelligent agents tackle real-world tasks, they require continuous knowledge updating to align with evolving world knowledge and dynamic user demands. To address this challenge, model editing methods update specific weights of Large Language Models (LLMs) without retraining; however, as edits accumulate, direct weight updates still lead to the catastrophic result of forgetting. In response, retrieval-based editing methods avoid modifying the original parameters directly by editing replicas and retrieving as well as activating the corresponding replicas during inference. Nevertheless, they remain confined to performing updates within a single layer and suffer from the coordination between editing and retrieval activation. To address these issues, we propose the Activation Control Layer-based continual Knowledge Editing (ACKE) method. First, we design a multi-layer replica framework that shares edits across FFN layers, thereby alleviating single-layer overload and reducing parameter merging frequency. Second, we devise an independent activation control layer integrated with an inverse similarity mechanism that decouples activation from editing, enabling more sensitive discrimination between edited and irrelevant knowledge for more robust retrieval. ACKE achieves an average score of 92.8% on ZsRE sequential editing and a Perplexity score of 1.88 on SelfCheckGPT. Compared with existing methods, it demonstrates stronger large-scale continual editing capability, providing a basis for more robust long-term knowledge updating of LLMs. The code is available at https://github.com/soraue/ACKE.