Performance Metrics of Binary Classifiers

In this chapter, we explore the impact of the prevalence threshold on several accuracy metrics of binary classification systems (BCS), notably, the F1 score, the \(F_\beta \) score, the Fowlkes-Mallows Index (FM) and the Matthews Correlation Coefficient (MCC), providing theorems in this regard. We further define the negative prevalence threshold, which is the equivalent threshold but for negative predictive value curves, \(\sigma (\phi )\) . Using a system of linear equations stemming from the \(2\times 2\) contingency table—we establish a matrix M, whose determinant is related to the accuracy of the system in question. Where computational resources and algorithmic time are a limiting resource, attaining the prevalence threshold in binary classification systems may be sufficient to yield levels of accuracy comparable to that under maximum prevalence.