Predicting Hospital Length of Stay (LOS) is essential for effective resource management, cost optimization, and improved patient care. Advances in machine learning (ML) and deep learning (DL) have significantly enhanced LOS prediction accuracy, yet these models often act as black-boxes, making their decisions difficult to interpret. To address this, we integrate both local and global Explainable AI (XAI) techniques to enhance transparency and interpretability. This study applies various ML models, ranging from interpretable to complex models, and fine-tunes them for optimal performance. The results show that combining XAI with ML/DL models improves both prediction accuracy and decision transparency. This approach bridges the gap between accuracy and interpretability, ensures trustworthiness and reliability.