The rapid advancement in artificial intelligence (AI) and natural language processing (NLP) has significantly influenced industrial automation and manufacturing operations. However, current AI models predominantly depend on cloud-based systems, which raises issues related to data privacy latency, and reliance on external servers. This research focuses on developing a modular on-premises Large Language Model (LLM)-based NLP system specifically designed for quality enhancement in the manufacturing sector. The proposed model facilitates real-time, context-sensitive, and secure analysis of machine operation protocols, inspection workflows, and standard operating procedures. The proposed model employs a retrieval-augmented generation (RAG) approach integrating FAISS-based vector store utilized for efficient document retrieval and Ollama’s open-source LLM for context-driven response generation. Multiple pre-trained models from the Ollama framework were tested for the semantic similarity scores, performance timing, memory profiling, and CPU profiling. It was observed that Lamma3 performed better comparatively with a slightly higher latency. The model was further evaluated to assess its effectiveness; it demonstrated a score of up to 0.92. However, the score was reduced when it could not incorporate the sub-steps in a particular response, as expected in the ground truth answer. The findings indicate that on-premises LLM was proficient in retrieving and generating informative suggestions in response to queries. An effective input prompt could further improve the response generation. Future research will explore the integration of multimodal AI, enabling the model to analyze multiple data types. Ultimately, the study establishes a foundation for intelligent, AI-driven manufacturing systems that improve quality monitoring and operational efficiency while safeguarding data sovereignty.