Smart Document Management: Harnessing Azure OpenAI’s Generative AI Chatbots to Boost Enterprise Efficiency
摘要
This release introduces an innovative interactive chatbot designed to engage with sensitive enterprise data. Leveraging Azure Machine Learning Promptflow and Retrieval-Augmented Generation (RAG) architecture, the chatbot facilitates secure data retrieval and generation within the enterprise environment. To assess the model’s performance, we utilized over 10 query examples, providing ground-truth and context data. Evaluation strategies included system-based metrics like the F1-Score, which yielded an average score of 0.59, and AI-evaluating-AI metrics such as Coherence, Groundedness, Similarity, Fluency, and Relevance, scoring 4.50, 4.20, 4.50, 4.10, and 4.40 respectively. While AI-evaluating-AI strategies showed decent scores, the relatively low F1-Score indicates potential for improvement through fine-tuning or selecting a more suitable vector database. Overall, this interactive solution not only enhances internal operations but also demonstrates AI’s potential in automating and streamlining complex processes.