AI (Artificial Intelligence) has provided many predictive algorithms for the diagnosis of many critical diseases. AI has also presented segmentation algorithms which can segment the desired area from the background for better diagnostic results. But AI-predictive algorithms suffer from AI-bias either due to training data or algorithmic design. This AI-bias leads to variability and inaccuracies in the predictive results which may have severe impact on treatment and clinical deployment of the model. Hence, it is necessary to evaluate the accountability of AI-bias in medical systems. Analysis of bias at various levels of AI-models in medical system design can prevent severity in the medical outcomes. In this chapter, we will highlight the bias accountability at various stages of AI-models. We will also review the various reasons and mitigation techniques to minimize the impact of AI-bias in medical systems.