The term “Explainable AI” (XAI) describes the capacity of AI systems to give humans intelligible justifications for their choices and actions. This is essential for fostering trust in AI, particularly in high-stakes domains like criminal justice, healthcare, or financial risk where choices have a direct influence on people’s lives. With the application of XAI approaches, AI models should become more accountable, transparent, and comprehensible so that users may comprehend the reasoning behind a given decision. XAI gives users a better understanding of the decision-making process, which increases user trust and facilitates more efficient usage of AI systems. Black-box machine learning models can be made interpretable with the help of XAI approaches like LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations). By approximating the model’s behavior around the prediction, LIME focuses on producing locally faithful explanations for specific forecasts. This is accomplished by first altering the input data and seeing how this affects the model’s output. Next, it fits an understandable model to these changes. Conversely, SHAP values take into account all potential feature permutations to offer a single, comprehensive estimate of feature relevance for every prediction. The Shapley value, which gives each characteristic a value according to how much it contributes to the prediction, is the foundation of cooperative game theory, which is where SHAP values originate. To improve openness and confidence in AI systems, LIME and SHAP are both helpful resources for deciphering and elucidating the choices made by intricate machine learning models. This paper focuses on the perspective of XAI with two experimental studies dealing with breast cancer detection and financial credit risk prediction.