A longstanding objective of multilingual neural machine translation (MNMT) is to develop models that maintain performance on previously learned language pairs while being sufficiently adaptable to incorporate new language pairs, without accessing previous training data. However, existing methods predominantly focus on mitigating catastrophic forgetting by compromising between original and new language pairs, prioritizing stability at the expense of plasticity. In this work, we propose a meta-contrastive memory replay (MCR) method to address the stability-plasticity trade-off in continual learning for MNMT. The method dynamically generates pseudo-samples of previous tasks based on its current state in the training process, thereby ensuring stability. Meanwhile, it boosts plasticity by leveraging contrastive learning within the meta-learning, which facilitates rapid adaptation to new tasks while minimizing inter-task interference. Experimental results demonstrate that MCR effectively adapts to new translation directions while preserving the performance of original tasks.