This release introduces an innovative interactive chatbot designed to engage with sensitive enterprise data. Leveraging Azure Machine Learning Promptflow and Retrieval-Augmented Generation (RAG) architecture, the chatbot facilitates secure data retrieval and generation within the enterprise environment. To assess the model’s performance, we utilized over 10 query examples, providing ground-truth and context data. Evaluation strategies included system-based metrics like the F1-Score, which yielded an average score of 0.59, and AI-evaluating-AI metrics such as Coherence, Groundedness, Similarity, Fluency, and Relevance, scoring 4.50, 4.20, 4.50, 4.10, and 4.40 respectively. While AI-evaluating-AI strategies showed decent scores, the relatively low F1-Score indicates potential for improvement through fine-tuning or selecting a more suitable vector database. Overall, this interactive solution not only enhances internal operations but also demonstrates AI’s potential in automating and streamlining complex processes.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Smart Document Management: Harnessing Azure OpenAI’s Generative AI Chatbots to Boost Enterprise Efficiency

  • Youssef Baklouti,
  • Tarik Echcherqaoui,
  • Ines Abdeljaoued-Tej

摘要

This release introduces an innovative interactive chatbot designed to engage with sensitive enterprise data. Leveraging Azure Machine Learning Promptflow and Retrieval-Augmented Generation (RAG) architecture, the chatbot facilitates secure data retrieval and generation within the enterprise environment. To assess the model’s performance, we utilized over 10 query examples, providing ground-truth and context data. Evaluation strategies included system-based metrics like the F1-Score, which yielded an average score of 0.59, and AI-evaluating-AI metrics such as Coherence, Groundedness, Similarity, Fluency, and Relevance, scoring 4.50, 4.20, 4.50, 4.10, and 4.40 respectively. While AI-evaluating-AI strategies showed decent scores, the relatively low F1-Score indicates potential for improvement through fine-tuning or selecting a more suitable vector database. Overall, this interactive solution not only enhances internal operations but also demonstrates AI’s potential in automating and streamlining complex processes.