This work proposes a systematic framework that employs eXplainable Artificial Intelligence (XAI) to conduct an in-depth comparative analysis of model explanations applied to breast cancer prediction. As the need for transparency and interpretability within Artificial Intelligence grows, our study delves into critical inquiries about the assessment of explanation quality, a comparative study of both direct and post-hoc explanation methods, and the trade-off between interpretability and explainability. The evaluation assists in carrying out well-rounded analyses of the explanations provided by the models. Our research not only enhances the understanding of explanation quality assessment but also underlines the required balance between interpretability and model efficiency, thereby revealing the correlation between model complexity and interpretability. Consequently, this framework serves as an efficient device for evaluating and selecting algorithms for breast cancer prediction, thereby encouraging more comprehensive utilization of Artificial Intelligence in clinical oncology.