Ultra-Dense Networks (UDNs) are a cornerstone of 5G, offering high-speed transmission and efficient resource management. However, managing frequent handovers in UDNs poses significant challenges, including increased handover failures and frequent triggering, which degrade user experience. This paper proposes an adaptive handover management approach using a multivariate Deep Q-Network (DQN) framework integrated with a Memory Anchor Repository (MAR) mechanism. The framework consists of three DQN models: \(\varvec{D}_\text {Dec}\) for handover decision-making, \(\varvec{D}_\text {TH}\) for adaptive adjustment of A2 and A4 thresholds, and \(\varvec{D}_\text {Tar}\) for target base station selection. These models leverage real-time features such as user location, movement direction, Signal-to-Interference-plus-Noise Ratio (SINR), and Reference Signal Received Power (RSRP). The MAR systematically stores and updates handover success rates at anchor points, enabling the system to learn from historical data and dynamically optimize handover decisions. Simulations conducted in a controlled UDN environment demonstrate that the proposed framework significantly reduces unnecessary handover attempts and failures. After 1250 training iterations, the overall handover failure rate decreases from 35% to 25%, with optimal performance observed using 25 anchor points. These results illustrate the framework’s potential to enhance UDN handover processes, improve overall Quality of Service (QoS), and elevate user experience.
