Continual learning (CL) [258] is a learning paradigm where an agent needs to continuously learn a sequence of tasks. To emulate the remarkable lifelong learning ability of humans, the agent is expected to leverage accumulated knowledge from previous tasks to more easily learn new ones, and further improve the learning performance of old tasks by leveraging the knowledge of new tasks. The former is referred to as forward knowledge transfer and the latter as backward knowledge transfer. One major challenge herein is the so-called catastrophic forgetting [231], i.e., the agent easily forgets the knowledge of old tasks when learning new tasks.