Women are the building blocks of the society. Improving women’s health should be of the utmost importance to any developing country. In recent days, different types of women’s cancer pose a great threat to the development of society. This happens due to the low awareness of women about the disease. In the digital era, a chatbot can be considered as a potential solution to serve this purpose. The primary challenge to implementing a successful chatbot is understanding the user’s query. Due to the low literacy levels, most Indian women provide incomplete or insufficient queries to the chatbot. As a consequence, they didn’t get the proper answers due to the misunderstanding of the queries by the artificial intelligence (AI) model. In this paper, we want to propose an LLM and Explainable AI-based chatbot that efficiently generates the complete queries from the input queries. In this study, we have considered XLNet, SHAP, and Integrated Gradients to develop the model. The incomplete/insufficient queries have been collected for surveying the students of Adamas University. The proposed model successfully achieves 94.95% accuracy which is quite acceptable for the community.