This paper presents a novel Continual Knowledge Graph Embedding (CKGE) approach that integrates local structural feature distillation to address the limitations of traditional static KGE methods. Our model dynamically updates embeddings without full retraining, reducing computational costs while retaining previously learned information. By employing a weighted sampling strategy based on betweenness centrality and a knowledge distillation mechanism that emphasizes entity-local structure similarity, our CKGE method effectively mitigates catastrophic forgetting. Experiments on four benchmark datasets reveal that our model surpasses existing baselines, showcasing its efficiency and effectiveness in continually learning knowledge graphs.