Recent advancements in Artificial Intelligence have enabled many telecommunication and e-commerce companies to leverage Large Language Model (LLM)-based chatbots to automate basic communication tasks and enhance customer support. However, a comprehensive understanding of user needs and insights is required before such agents can be effectively implemented. This understanding is typically achieved by analysing past user feedback data such as inquiries, reviews and feedback. Here, the challenge is to extract meaningful knowledge (i.e. identifying user insights and pertinent questions), especially when the volume of reviews exceeds the human capacity for manual processing and analysis. Consequently, there is a demand for an automated solution that discovers insightful user questions to create a knowledge base useful for chatbot development. This paper proposes an unsupervised framework (TopInsight) to find and generate diverse, high-quality, insightful questions from extensive user feedback data by leveraging topic models and LLMs. TopInsight involves a three-step process: knowledge discovery using topic models, topic-guided document retrieval, and question generation via LLMs. The effectiveness of this framework is evaluated through extensive experimentation on three open-source review datasets. We also demonstrate the use of TopInsight through a case study of an Australian telecom company using its conversation dataset.