<p>The rapid expansion of Artificial Intelligence (AI) applications necessitates the introduction of statistical methods and metrics that can assess their quality, not only from a technical viewpoint (accuracy, sustainability); but also from an ethical viewpoint (explainability, fairness). In this paper, we contribute to fill the gap proposing a set of consistent statistical metrics to measure the Sustainability, Accuracy, Fairness and Explainability of AI applications, integrated in an open-source Python package, which allows their full reproducibility. They are easy to interpret, as are all expressed in percentages of an ideal situation of full compliance. They are agnostic, as they can be applied to any Machine Learning method. They are fully reproducible, by means of the proposed Python safeaipackage, which serves as a convenient development environment for Python programmers.</p>