Artificial intelligence and machine learning algorithms are increasingly driving many decisions that can impact several aspects of human life, including healthcare, college admissions, recruitment, and loan provision, among others. Given their extensive influence on our daily experiences, it is imperative to adopt algorithms that are aware of the uncertainty in their predictions and of the discrimination that they can induce. This article explores various methods for quantifying uncertainty in fairness metrics. Furthermore, it introduces a modification to the CART algorithm designed to improve fairness in binary classification tasks by incorporating uncertainty in these metrics. This modified algorithm adaptively applies penalties to unfair splits, thereby enhancing prediction fairness while preserving a high level of accuracy.