Unbalanced Datasets and Machine Learning Metrics

This chapter explores the challenges and solutions associated with unbalanced datasets in machine learning. It begins by defining what constitutes an unbalanced dataset and emphasises their prevalence. The chapter introduces the concept of machine learning metrics, vital for evaluating the performance of models trained on such datasets. It presents a simple example to illustrate how traditional models fail in the face of extreme class imbalances and how this leads to misleading accuracy metrics. The core of this chapter delves into various approaches to address dataset imbalance, focusing primarily on data level approaches like oversampling and undersampling. It discusses the advantages and disadvantages of these techniques and highlights their practical applications through examples. The Synthetic Minority Oversampling Technique (SMOTE) is introduced as a sophisticated method to generate synthetic samples to balance datasets. Moreover, the chapter covers crucial metrics for assessing model performance in the context of unbalanced datasets, including the confusion matrix, sensitivity, specificity, precision, Fβ-score, and balanced accuracy. The discussion extends with the Receiving Operating Characteristic (ROC) curve and the Area Under the Curve (AUC), providing a comprehensive framework for evaluating and enhancing model performance in situations of class imbalance.