A classifier expansion framework with dual knowledge distillation and dynamic weighting for continual relation extraction
摘要
Continual relation extraction (CRE) aims to learn new relations while avoiding catastrophic forgetting of old ones. Memory-based methods mitigate forgetting by replaying samples stored in a memory buffer, but performance drops significantly without the buffer. This suggests these methods rely heavily on memory replay and neglect the stability of retaining learned knowledge. Repeatedly replaying simple samples may also cause overfitting. To address these issues, we propose a classifier expansion framework based on dual knowledge distillation and dynamic weighting. This method incrementally expands the classifier and uses dual knowledge distillation to retain old knowledge while learning new relations. We design a forgetting weight and an adaptation loss to balance distillation loss. Additionally, we weight classification loss by relation difficulty to prioritize challenging relations and reduce overfitting on simpler ones. Experimental results show that our method effectively reduces forgetting and improves model performance by enhancing stability and mitigating overfitting.