The consensus on what evaluation metric to be used to measure the results of the binary classification is not reached in the computer science because the researches’ conclusions often contradict each other while recommending the metric. The goal of this study is to propose the approach on how to obtain the maximum expected pair scores difference for metrics: Matthews Correlation Coefficient, Balanced Accuracy, Cohen’s Kappa and how to use those values to support a decision-making process on what evaluation metric to select to evaluate binary classification results. The approach to be used when it is important that the model doesn’t make mistakes while predicting the negative instances. To achieve the goal of the research were performed: study of relations between formal description of Youden Index, Matthews Correlation Coefficient, Balanced Accuracy and Cohen’s Kappa; identified the conditions when the metrics have the maximum expected scores and therefore the maximum difference in their pair scores; executed tests to receive the practical evidences.