This study derives analytical bit error rate (BER) expressions for the downlink multiple-input-multiple-output (MIMO) non-orthogonal multiple access (NOMA) system over the generalized \(\alpha -\kappa -\mu \) fading channel. The expressions are formulated using the upper bound of the Q-function and the moment generating function for two scenarios—erroneous and error-free successive interference cancellation. Simulations are conducted to validate the derived expressions, and the strong agreement between analytical and simulation-based BER curves substantiates the accuracy of the theoretical formulations. To assess the generalization capability of the \(\alpha -\kappa -\mu \) fading model, its simulation and empirical probability density functions (PDFs) are compared with the theoretical PDFs of standard fading distributions, namely Rayleigh and Nakagami-m. Results also demonstrate that MIMO-NOMA achieves significantly better BER performance than conventional NOMA. Overall, the findings confirm the effectiveness of the proposed \(\alpha -\kappa -\mu \) based MIMO-NOMA framework in accurately modeling diverse wireless environments and enhancing communication reliability.
