Reinforcement learning Double Deep Q-Network Techniques for NextG Network
摘要
NextG aims for ultra-fast, reliable, and intelligent wireless connectivity, surpassing 5G’s capabilities, enabling emerging applications like holographic communication, tactile internet, and Machine Learning (ML) services, and achieving high performance. In NextG wireless systems, conventional massive Multiple-Input Multiple-Output Orthogonal Frequency Division Multiplexing (mMIMO-OFDM) relies on static transmission parameters. Additionally, Reinforcement Learning (RL) techniques, like Double Deep Q-Network (DDQN), are underexplored for integrating strategies to optimize power allocation and subcarrier selection, resulting in degraded performance in terms of Spectral Efficiency (SE), Energy Efficiency (EE), and Bit Error Rate (BER). This study presents a framework for utilizing specifically DDQN-based RL, leveraging a Deep Neural Network (DNN) and Markov Decision Process (MDP). The DDQN agent enables efficient power and subcarrier distribution using an epsilon-greedy policy with optimal parameter selection for dynamic wireless communication optimization in mMIMO-OFDM systems. The dynamic system integrates with diverse modulations to enhance SE, EE, and BER, thereby setting it apart from conventional static systems. The MATLAB simulations in this research corroborated the methodology through extensive evaluations across various QAM orders and mMIMO 128 and 512, demonstrating a DDQN enhancement for SE (21.3% and 28.6%), an EE (15.7% and 19.1%), and a (1-1.5 dB) advantage at a BER of ≤ 10⁻⁵. This study demonstrates the ability of ML algorithms to adapt to wireless settings, providing real-time optimization for NextG networks. In addition, research lays the groundwork for ML-enhanced frameworks that support sustainable and scalable NextG networks, thereby enhancing performance and adaptability.