Text-guided bidirectional mapping distillation for continual semantic segmentation

Continual semantic segmentation (CSS) has been extensively studied to address the challenge of catastrophic forgetting, which refers to the significant drop in a model’s performance on old classes when learning new ones. Recently, CSS methods have mainly utilized knowledge distillation to tackle this problem. However, most distillation-based methods directly constrain the output of the new model to be similar to that of the old model, overlooking the need for plasticity. In this work, we explore a semantic relationship between old and new classes, which can be applied to facilitate knowledge transfer. Specifically, we leverage the text embeddings of image-level labels to model this relationship. Guided by this semantic relationship, we introduce a novel bidirectional mapping distillation that transfers old knowledge forward to facilitate learning new classes while transferring new knowledge backward to resist forgetting. By employing the text-guided bidirectional mapping distillation, our model achieves a better trade-off between stability and plasticity. Extensive experiments on the PASCAL VOC 2012 and ADE20K datasets under various CSS scenarios demonstrate that our method achieves competitive performance.