Prompt-Based Hypernetwork in Rehearsal-Free Class Incremental Learning

Among the three scenarios of continual learning, task incremental learning is considered an ideal scenario under certain assumptions, where the model is assumed to have access to task identifiers during the inference phase. In this scenario, numerous advanced methods have emerged, among which hypernetwork-based approaches stand out. These methods use an auxiliary network to generate weights or masks for the main network, serving to protect the weights and achieving favorable results. However, a limitation of this approach is that, due to the task-specific nature of the hypernetwork, task identifiers are essential. To address this limitation, we introduce prompt matching mechanism into the hypernetwork, enabling this approach to be applied in class-incremental learning without recourse to a task identifier. Furthermore, we combine the prompt-based hypernetwork with prompt-tuning methods to explore additional functionalities of prompts. Experimental results show that our prompt-based hypernetwork experiences almost no performance degradation in the class incremental learning scenario. Additionally, when our hypernetwork is used as a plug-in for prompt-tuning methods in class-incremental learning, it improves accuracy by 1.0–1.5% and achieves state-of-the-art (SOTA) results on several continual learning benchmarks.