Forecasting student academic performance is a crucial aspect of intelligent education systems supported by computer technology. Early identification of low-performing students is vital for faculty and administration to offer timely support and analyze the results to make decisions at an early stage. This study uses preprocessing techniques and data mining algorithms to predict student performance at the initial stage. Academic data has been obtained from the SIS of the Faculty of Information Technology at Sebha University, while some Demographic information and family background have been collected using a questionnaire. The data is classified into two classes based on CGPA: At risk (<50%) and not at risk (> = 50%). The process involves data cleaning, normalization, and feature selection methods using RFE with a Random Forest (RF) classifier and Support Vector Machine (SVM) to obtain important features, while SMOTE has been used to address imbalanced data. Subsequently, ten classification algorithms—including Decision Tree (DT), Naïve Bayes (NB), k-nearest Neighbor (k-NN), and SVM—were employed, along with an ensemble method known as RF, for the prediction task. The results show that the RF classifier has higher Results with a Full Dataset and Balanced dataset of 80.63% and 79.30%, respectively. In contrast, the Logistic Regression classifier has higher accuracy with the Normalized dataset, Dataset of RFE(RF), and RFE(SVM) of 80.63%, 81.69%, and 81.97%, respectively.