<p>Massive-multiple input and Multiple Outputs Non orthogonal multiple access (M-MIMO–NOMA) systems require efficient signal detection techniques to mitigate interference and enhance spectral efficiency, especially under diverse channel conditions and varying modulation schemes. This study investigates the performance of the Successive Interference Cancellation with Reinforcement Learning (SIC-RL) detector compared to conventional methods, including the Minimum Mean Square Error (MMSE), Maximum Likelihood Detection (MLD), Approximate Message Passing (AMP), Gauss–Seidel (GS), Conjugate Gradient (CG), and zero-forcing equalizer (ZFE). The analysis was conducted for 16-QAM, 64- Quadrature Amplitude Modulation (QAM), and 256-QAM in Rayleigh fading channels with 10% error. The simulation results indicate that SIC-RL outperforms traditional detectors in terms of bit error rate (BER), power spectral density (PSD), and computational complexity. At a BER of 10⁻<sup>3</sup>, SIC-RL achieves an SNR 11.2 dB (512-QAM), 6.6 dB (256-QAM), 5 dB (64-QAM) and 5.8 dB (64-QAM with 10% channel error) as compared with conventional methods. The PSD analysis shows that SIC-RL exhibits a 35% and 20% lower spectral leakage compared to contemporary methods, ensuring better spectral efficiency for diverse channel conditions. In terms of computational complexity, SIC-RL achieves near-logarithmic growth with the number of antennas, significantly reducing the processing burden compared to MLD, which has exponential complexity. Although ZFE and CG are computationally efficient, they suffer from noise amplification and poor BER performance. GS and AMP balance complexity and performance but still fall short of SIC-RL gains. Overall, SIC-RL has emerged as an optimal solution for massive MIMO signal detection, achieving a superior trade-off between BER, PSD, and computational efficiency across diverse modulation schemes and channel conditions. Critically, SIC-RL achieves near-quadratic complexity <InlineEquation ID="IEq1"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="41598_2025_6492_Article_IEq1.gif" Format="GIF" Height="20" Rendition="HTML" Resolution="72" Type="Linedraw" Width="54" /> </InlineMediaObject> <EquationSource Format="TEX">\(O(N_{t}^{2} ),\)</EquationSource> </InlineEquation> contrasting the exponential complexity <InlineEquation ID="IEq2"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="41598_2025_6492_Article_IEq2.gif" Format="GIF" Height="21" Rendition="HTML" Resolution="72" Type="Linedraw" Width="60" /> </InlineMediaObject> <EquationSource Format="TEX">\(O(M^{{N_{{\text{t}}} }} )\)</EquationSource> </InlineEquation> of MLD and the cubic complexity <InlineEquation ID="IEq1111"> <InlineMediaObject> <ImageObject Color="BlackWhite" FileRef="41598_2025_6492_Article_IEq1111.gif" Format="GIF" Height="20" Rendition="HTML" Resolution="72" Type="Linedraw" Width="49" /> </InlineMediaObject> <EquationSource Format="TEX">\(O(N_{t}^{3} )\)</EquationSource> </InlineEquation> of MMSE and ZFE, making it scalable for large antenna arrays. Iterative methods such as AMP, GS, and CG achieve lower complexity, but suffer from convergence issues or degraded BER. Despite requiring initial training, SIC-RL provides a favorable trade-off between the detection accuracy and processing cost, positioning it as a computationally efficient and high-performance detector for next-generation MIMO–NOMA systems.</p>

SIC based RL for massive MIMO NOMA signal detection for different modulation schemes under diverse channel conditions

期刊论文
出版模式：

开放获取
发表日期：

2025年7月9日

查看全文

Arun Kumar,
Aziz Nanthaamornphong,
Mohammed H. Alsharif,
Mehedi Masud

摘要

Massive-multiple input and Multiple Outputs Non orthogonal multiple access (M-MIMO–NOMA) systems require efficient signal detection techniques to mitigate interference and enhance spectral efficiency, especially under diverse channel conditions and varying modulation schemes. This study investigates the performance of the Successive Interference Cancellation with Reinforcement Learning (SIC-RL) detector compared to conventional methods, including the Minimum Mean Square Error (MMSE), Maximum Likelihood Detection (MLD), Approximate Message Passing (AMP), Gauss–Seidel (GS), Conjugate Gradient (CG), and zero-forcing equalizer (ZFE). The analysis was conducted for 16-QAM, 64- Quadrature Amplitude Modulation (QAM), and 256-QAM in Rayleigh fading channels with 10% error. The simulation results indicate that SIC-RL outperforms traditional detectors in terms of bit error rate (BER), power spectral density (PSD), and computational complexity. At a BER of 10⁻³, SIC-RL achieves an SNR 11.2 dB (512-QAM), 6.6 dB (256-QAM), 5 dB (64-QAM) and 5.8 dB (64-QAM with 10% channel error) as compared with conventional methods. The PSD analysis shows that SIC-RL exhibits a 35% and 20% lower spectral leakage compared to contemporary methods, ensuring better spectral efficiency for diverse channel conditions. In terms of computational complexity, SIC-RL achieves near-logarithmic growth with the number of antennas, significantly reducing the processing burden compared to MLD, which has exponential complexity. Although ZFE and CG are computationally efficient, they suffer from noise amplification and poor BER performance. GS and AMP balance complexity and performance but still fall short of SIC-RL gains. Overall, SIC-RL has emerged as an optimal solution for massive MIMO signal detection, achieving a superior trade-off between BER, PSD, and computational efficiency across diverse modulation schemes and channel conditions. Critically, SIC-RL achieves near-quadratic complexity \(O(N_{t}^{2} ),\) contrasting the exponential complexity \(O(M^{{N_{{\text{t}}} }} )\) of MLD and the cubic complexity \(O(N_{t}^{3} )\) of MMSE and ZFE, making it scalable for large antenna arrays. Iterative methods such as AMP, GS, and CG achieve lower complexity, but suffer from convergence issues or degraded BER. Despite requiring initial training, SIC-RL provides a favorable trade-off between the detection accuracy and processing cost, positioning it as a computationally efficient and high-performance detector for next-generation MIMO–NOMA systems.