Scholarlink

<p>To consolidate previous knowledge, various rehearsal strategies have been proposed in the existing class incremental learning (CIL) methods. Albeit achieving promising performance, the underlying rationality has not been fully discussed. In this study, motivated by the potential mismatch of batch normalization (BN) statistics between training and inference phases, we analyze the BN technique adopted in these rehearsal strategies. Specifically, through exploratory experiments and detailed analysis, we investigate the <i>BN discrepancy</i> issue and elaborate its underlying causes and influences. We find that although such BN discrepancy may degrade testing performance, it is potential to help alleviate the ubiquitous class imbalance problem in rehearsal-based CIL models. Inspired by such observations, we propose a novel rehearsal strategy named <b>S</b>plit with s<b>T</b>abilized <b>EM</b>A <b>S</b>tatistics (<b>STEMS</b>), to regulate the updating of BN statistics and reduce the classification bias for rehearsal-based CIL methods. Through comprehensive experiments conducted on various benchmark CIL datasets with disjoint and blurry task boundaries, we show that STEMS can bring significant performance gains to various rehearsal-based CIL methods, revealing its potential generality along this line of research.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Diagnosing BN discrepancy in rehearsal-based class incremental learning

  • 期刊论文

  • 发表日期：

    2026年2月28日

查看全文
  • Minghao Zhou,
  • Quanziang Wang,
  • Renzhen Wang,
  • Jun Shu,
  • Qian Zhao,
  • Hong Wang,
  • Deyu Meng

摘要

To consolidate previous knowledge, various rehearsal strategies have been proposed in the existing class incremental learning (CIL) methods. Albeit achieving promising performance, the underlying rationality has not been fully discussed. In this study, motivated by the potential mismatch of batch normalization (BN) statistics between training and inference phases, we analyze the BN technique adopted in these rehearsal strategies. Specifically, through exploratory experiments and detailed analysis, we investigate the BN discrepancy issue and elaborate its underlying causes and influences. We find that although such BN discrepancy may degrade testing performance, it is potential to help alleviate the ubiquitous class imbalance problem in rehearsal-based CIL models. Inspired by such observations, we propose a novel rehearsal strategy named Split with sTabilized EMA Statistics (STEMS), to regulate the updating of BN statistics and reduce the classification bias for rehearsal-based CIL methods. Through comprehensive experiments conducted on various benchmark CIL datasets with disjoint and blurry task boundaries, we show that STEMS can bring significant performance gains to various rehearsal-based CIL methods, revealing its potential generality along this line of research.