Continual learning aims to train models across multiple tasks, each containing different knowledge. The biggest challenge in continual learning is that models quickly lose previously learned knowledge when acquiring new tasks, as they lack access to their old knowledge. Generalized Class Incremental Learning (GCIL) poses the challenge that knowledge may not be uniform and may overlap between tasks. Some current methods store some old knowledge to prevent the model from forgetting, but this can violate data security. This research proposes the ZeroRFF (Zero-Memory Random Fourier Features) method, which solves the generalized class incremental learning problem without storing old knowledge (exemplar-free). This method uses analytic learning techniques; instead of relying on gradient descent, ZeroRFF finds optimal solutions by directly solving equations where the derivative equals zero. Additionally, ZeroRFF integrates Random Fourier Features (RFF) to transform linearly non-separable data into a new space where the data becomes linearly separable, enhancing the model’s classification capability. Experimental results on the CIFAR100 dataset show that the ZeroRFF method achieves superior performance compared to other methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ZeroRFF: A Random Fourier Features and Analytic Learning Method for Generalized Class Incremental Learning

  • Duc-Hung Nguyen,
  • Tri-Thanh Nguyen,
  • Thanh-Hai Dang,
  • Quynh-Trang Pham Thi

摘要

Continual learning aims to train models across multiple tasks, each containing different knowledge. The biggest challenge in continual learning is that models quickly lose previously learned knowledge when acquiring new tasks, as they lack access to their old knowledge. Generalized Class Incremental Learning (GCIL) poses the challenge that knowledge may not be uniform and may overlap between tasks. Some current methods store some old knowledge to prevent the model from forgetting, but this can violate data security. This research proposes the ZeroRFF (Zero-Memory Random Fourier Features) method, which solves the generalized class incremental learning problem without storing old knowledge (exemplar-free). This method uses analytic learning techniques; instead of relying on gradient descent, ZeroRFF finds optimal solutions by directly solving equations where the derivative equals zero. Additionally, ZeroRFF integrates Random Fourier Features (RFF) to transform linearly non-separable data into a new space where the data becomes linearly separable, enhancing the model’s classification capability. Experimental results on the CIFAR100 dataset show that the ZeroRFF method achieves superior performance compared to other methods.