With the rapid growth of artificially intelligent algorithms, scientists have begun to utilize them in healthcare for decision support systems. The advancement and refinement of these algorithms improve the accuracy of these systems tremendously. The system is accurate but hides many aspects, like how it reaches the decision. The black box behavior of the decision support system makes it untrustworthy, especially in the medical domain. The AI-based disease diagnosis system requires the correct explanation of the achieved result. The lack of explanation turns into an untrusted system and makes the treatment step difficult. The presented work provides an interpretation of the chronic kidney disease diagnosis system. A logistic regression classifier has been trained on the UCI KDD chronic kidney disease dataset. To explain system decisions locally and globally, a coalition game theory-inspired Shapley additive explanations method is used. The result suggests that the features of serum creatinine, hemoglobin, blood glucose at random, albumin, and blood pressure prove to be very crucial for deciding the chance of chronic kidney disease.