In the dynamic realm of face recognition systems, the perpetual challenge of accommodating facial variations over time remains a significant hurdle to achieving accurate identification, particularly in real-world applications. Traditional methods encounter difficulties in adapting to these changes, primarily due to their reliance on static reference galleries. Such static repositories inhibit the system’s capacity to effectively adjust to temporal and cosmetic alterations in facial features, including aging, changes in hairstyle, facial hair growth, and cosmetic enhancements. To address this critical issue, this paper introduces a novel approach integrating Reinforcement Learning (RL) with Deep Learning (DL)-based face recognition systems. Our method focuses on manipulating reference representations through RL, enabling efficient self-supervised continual learning. By dynamically updating reference databases and adapting reference representations in real-time, our approach demonstrates significant enhancements in face recognition accuracy, even in scenarios with substantial variations in facial features. Leveraging RL empowers our system to learn and adapt to these changes iteratively, ensuring its effectiveness across different temporal stages and environmental conditions. Through extensive evaluation on diverse datasets, we validate the effectiveness of our proposed method, highlighting its potential to advance face recognition technology in real-world settings. By addressing the limitations of traditional methods and providing a robust solution capable of handling dynamic facial variations, our approach contributes to the progression of the field, opening new avenues for deploying more resilient and reliable face recognition systems in various practical applications.