Failure of rolling element bearings (REBs) can cause hazardous effects on rotating machinery. When bearings fail, they may lead to increased vibration, excessive heat, and even catastrophic machine breakdowns, potentially risking both equipment and personnel. Therefore, the diagnosis of bearing faults is crucial for maintaining the reliable operation of such machinery. The machine learning (ML) process for this involves three main steps: feature extraction, selection, and classification. Feature selection focuses on identifying the most effective features that enhance classification accuracy while reducing the number of features and computational time. For large feature dimensions, where the dataset contains a vast number of features, it becomes imperative to conduct a critical study to determine the optimal feature subset. This is necessary to ensure proper diagnosis and maintain high classification performance. This study explores how the performance of decision tree model in fault classification of REBs is affected by the application of the Minimal-Redundancy-Maximal-Relevance (MRMR) feature selection technique. It utilizes the open-source bearing datasets from CWRU (Case Western Reserve University) for this purpose. The impact of ranking the features on the diagnostic capability of the ML classifier is assessed by examining metrics such as classification accuracy, training time, and prediction speed. It is observed that the process of choosing exclusive features via feature ranking evidently affects the efficiency of the ML model. The highest level of accuracy in fault classification is attained by utilizing a combination of top-ranked time-domain and frequency-domain features with the support of the MRMR feature selection scheme.