Cross Classification Matrix to Evaluate the Performance of Machine Learning Algorithms in Predicting Students Performance of Developing Regions

In the rapidly evolving landscape of education, the integration of Big Data and AI presents significant opportunities for improving educational outcomes, especially in the context of Predicting Students Performance (PSP) applications in higher education. Today, Educational Data Mining (EDM) strategies have been implemented to overcome educational challenges in advanced nations. Nonetheless, the issues confronting developing countries, like the unavailability of educational datasets, and challenges with selecting Machine Learning (ML) algorithms that are effective in terms of accuracy, bias, and over-fitting, have never been investigated. Therefore, a novel dataset, UOBEDM, collected from the University of Baluchistan (UoB) in the developing region of Balochistan, Pakistan, comprises 49,835 student records, providing valuable insights into various demographic and academic aspects. Through meticulous data collection and cleaning processes, including feature selection techniques, the dataset was refined to 23,492 instances. Various ML algorithms were fine-tuned on the UOBEDM dataset, with the top five algorithms—Trees, K-Nearest Neighbors (KNN), Naive Bayes (NB), Random Forest (RF), and Support Vector Machines (SVM)—yielding accuracy scores of 0.95, 0.94, 0.92, 0.96, and 0.50, respectively. A novel approach called the Cross-Classification Matrix (CCM) was introduced to assess algorithm performance and select the best model. Trees emerged as the optimal predicting algorithm, simplifying decision-making processes for academics through the development of a graphical tree-based Early Intervention Model (EIM). The significance of the dataset extends beyond classification algorithms, paving the way for research in EDM and addressing educational inequalities. This study underscores the potential of data-driven approaches to enhance educational outcomes and foster innovation in education. The findings contribute to the understanding of predictive modeling in education and provide valuable insights for educators, policymakers, and researchers.