In geotechnical engineering, slope stability analysis has been of long-lasting interest to ensure the safety of embankments. A wide range of theoretical, numerical, and experimental techniques has been applied extensively to calculate the factor of safety, while the use of intelligence-based approaches has promised a new paradigm shift with an ever-increasing application. However, the black-box nature of artificial intelligence (AI)-driven tools, in which the internal mechanism in making decisions by the AI-centered decision support systems is not clear even for developers, has a damper effect on its usage where the consequences can affect human safety. This paper addresses the problem of lack of reliability and transparency in the machine learning (ML)-assisted safety factor prediction of embankments. A dataset of different slopes is tainted via a group of ML techniques automatically. Then, the best-tuned one is utilized for explainable AI (XAI)-powered exploration of input parameters. In particular, Shapley Additive exPlanation (SHAP) analysis allowed for understating about the most important contributing factors and their interactions in an interpretable way. This study prepares the ground for applying intelligent safety factor prediction tools in a reliable and transparent style.