Major Depressive Disorder is a widespread psychiatric disorder that has an emotional impact on numerous individuals globally. Electroencephalography offers a non-intrusive and objective approach to recognizing brain activity patterns linked with depression in mental health diagnoses. The study conducts a detailed review of different machine learning classifiers— Decision Tree, Random Forest, Support Vector Machine, and K-Nearest Neighbors—by analyzing power spectral density in the frequency domain and Hjorth parameters in the time domain on a private dataset of depressed and healthy individuals. The performance was evaluated by 5-fold and 10-fold cross-validation methods integrating Hyperparameter optimization to enhance the generalizability of the model. The results demonstrate that Decision Tree outperformed other methods and showed an extremely high accuracy of 98% across all feature sets in 10-fold cross-validation. Both Random Forest and Support vector machine classifiers also delivered reliable performance whereas K-Nearest Neighbor provides inferior results with power spectral density features having high standard deviation in metrics. Notably, Hjorth parameters are more reliable and effective as they accomplish high F1 scores on each classifier than other features. One-way ANOVA tests were conducted on the F1-scores to check whether the performance differences among multiple classifiers were statistically significant across different feature sets. These findings emphasize the critical role of feature selection and the efficacy of each classifier in EEG-based depression detection by flagging the methodology for more accurate and dependable diagnostic tools in clinical mental health applications.