Exploring Evaluation Metrics for Binary Classification in Data Analysis: the Worthiness Benchmark Concept

In binary data classification, the main goal is to determine if elements belong to one of two classes. Various metrics assess the efficacy of classification models, making it essential to analyze and compare these metrics to select the most appropriate one. Despite significant research, a comprehensive comparison of these metrics has not been adequately addressed. The effectiveness of classification models is typically represented by a confusion matrix, detailing the count of correct and incorrect predictions for each class. Evaluating changes in the confusion matrix is crucial to discern model superiority, but different metrics may yield varying interpretations of the same matrices. We propose the Worthiness Benchmark ( \(\gamma \) ), a novel concept characterizing the classification metrics’ principles for ranking classifiers and is useful to select the best metric for a given problem.