A Unified Approach for Identification and Analysis of the Sources of Uncertainty in Machine Learning Techniques

Over the last decade, there has been an intensive effort to introduce technologies that increase system smartness by incorporating artificial intelligence and machine learning, resulting in a better way of life. With this evolution, it has become particularly important to understand how trustworthy are the decisions made by autonomous systems driven by artificial intelligence coupled with machine learning and deep learning capabilities. Uncertainty quantification (UQ) has thus become a topic of interest over the last few years. In this paper, we observe the behavior of the epistemic (model), aleatoric (data), and distribution uncertainty and deduce a mathematical relationship between them. Next, we show that the epistemic probability function is inversely proportional to the distribution probability function and is directly proportional to the aleatoric probability function. Moreover, we demonstrate the impact of identifying in-domain distribution and out-of-domain distribution in model and data uncertainties. Finally, we perform an array of experiments using lung cancer data to demonstrate that improved accuracy may be obtained by striking a correct balance between the three forms of uncertainty. According to experimental findings, methodical data selection aids in adopting informed decisions, strengthening the system's dependability and enabling it to achieve close to 99% accuracy even with basic machine learning models.