Scholarlink

<p>Continual Learning (CL) has proven an effective method of avoiding Catastrophic Forgetting (CF) when sequentially training neural networks. This improves network efficiency while facilitating the Knowledge Transfer (KT) between tasks. CL serves as an ideal setting for studying KT and network behavior. In particular, pruning methods for CL train subnetworks to handle the sequential tasks which allows us to take a structured approach to investigating KT. This subnetwork approach allows for the promotion of Forward KT (FKT) and complete prevention of negative Backward KT (BKT) and CF. Understanding which weights to share between tasks is crucial for FKT as sharing all weights can worsen accuracy. This paper demonstrates how the information flow (IF) within the network can reflect task similarity and usefulness, informing optimal sharing decisions to improve FKT between diverse tasks. We leverage IF to determine which weights to share to promote FKT. By doing this, we match the optimal sharing decisions for each implemented dataset, and compare the resulting accuracies against several benchmark methods which emphasize promoting FKT and BKT. Experiments are run on three CL datasets designed to emphasize variation in task complexity and similarity, using both ResNet-18 and VGG-16.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Leveraging Information Flow for Knowledge Transfer in Continual Learning

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月22日

查看全文
  • Joshua Andle,
  • Ali Payani,
  • Salimeh Yasaei-Sekeh

摘要

Continual Learning (CL) has proven an effective method of avoiding Catastrophic Forgetting (CF) when sequentially training neural networks. This improves network efficiency while facilitating the Knowledge Transfer (KT) between tasks. CL serves as an ideal setting for studying KT and network behavior. In particular, pruning methods for CL train subnetworks to handle the sequential tasks which allows us to take a structured approach to investigating KT. This subnetwork approach allows for the promotion of Forward KT (FKT) and complete prevention of negative Backward KT (BKT) and CF. Understanding which weights to share between tasks is crucial for FKT as sharing all weights can worsen accuracy. This paper demonstrates how the information flow (IF) within the network can reflect task similarity and usefulness, informing optimal sharing decisions to improve FKT between diverse tasks. We leverage IF to determine which weights to share to promote FKT. By doing this, we match the optimal sharing decisions for each implemented dataset, and compare the resulting accuracies against several benchmark methods which emphasize promoting FKT and BKT. Experiments are run on three CL datasets designed to emphasize variation in task complexity and similarity, using both ResNet-18 and VGG-16.