The traditional sarees in India represent a rich history both culturally and artistically, as their patterns are created from regional influences along with modern fashion. Making sarees requires detailed skill, and traditional techniques are highly laborious and time-consuming. In this research, we have developed Vastra Kalpana, an AI-driven saree design generator that uses generative deep learning models to automate textile pattern creation. Users can now provide voice commands, and they are converted to text prompts by our integrated OpenAI Whisper speech-to-text software, which are then transformed into structured textual descriptions. These descriptions serve as instructions for the Stable Diffusion’s high-resolution saree design generator. Our research results suggest that this automated saree design generator is both solution oriented and efficient, proving the generative techniques offer a novel approach for saree design while tackling the challenges of overreliance on handmade designs. This research mainly contributes to the field of fashion design and establishes a framework for future advancements in automated textile pattern generation.