A primary objective of current Continual Semantic Segmentation (CSS) is to solve the problem of catastrophic forgetting. One common approach involves using pseudo-labels generated from the predictions of previous models as a reference for new model training. Typically, an entropy threshold is set during training, and pseudo-labels with entropy above this threshold are considered unreliable and therefore discarded. We contend that such pixels can still provide valuable information for training. Although it is often inaccurate to use an old model’s predictions to determine which category unreliable pixels belong to, it is generally reliable for determining which categories they do not belong to. Consequently, we implement selective contrastive learning, designating such pixels as negative exemplars for their respective low-probability categories, thus distilling pertinent information. To accelerate the training process, we optimize using the knowledge distillation method. Furthermore, to enhance the integration of the selective contrastive learning method into Continual Semantic Segmentation, we propose an innovative strategy to dynamically adjust the threshold for selecting pseudo-labels as the steps progress. Additionally, we propose a novel approach for negative sample selection, focusing contrastive learning on samples with relatively small differences yet greater challenges, aiming to boost the efficiency of contrastive learning. Upon experimentation with widely utilized datasets, our method showed a discernible improvement over the current baseline.