Query systems in higher education institutions are in need of improvement to enhance communication, response times, and access to information. To address this pressing need, this research explores a novel approach to integrating large language models (LLMs) with custom knowledge bases specifically for higher education institution. We collected data from Sheth L. U. J. College of Arts and Sir M. V. College of Science and Commerce in Mumbai, India, in both structured and unstructured formats, following all higher education guidelines. Using the Llama-Index model for text embedding and OpenAI GPT-3.5 for response generation, our system achieved a remarkable average response time of 5 seconds and an average relevancy score of 0.93 across various query categories. These results unequivocally demonstrate the potential of LLM-powered systems to streamline interactions and provide exceptionally useful, relevant data in higher education settings. We strongly advocate for the implementation of these systems to significantly enhance the user experience and communication within the higher education landscape, even while acknowledging their potential cost implications. Looking ahead, our research roadmap includes the expansion of knowledge sources and the integration of open-source LLM models to further augment the effectiveness and coverage of these systems.