Scholarlink

<p>Many organizations still face significant challenges in document audits, such as contract reviews, accounting audits, and compliance checks, which require substantial manpower and time. Current Artificial Intelligence solutions struggle with ambiguous requirements, unclear standards, and complex documents, limiting their scalability and effectiveness. To address these challenges, we propose a novel architecture, LLMES, designed to generate a concise and independent audit checklist. This approach allows Large Language Models to perform audits and produce results without prior fine-tuning. LLMES effectively integrates complex legal documents, expert knowledge, and mandatory regulatory guidelines with large language modeling to enhance artificial intelligence-assisted auditing. In our experiments auditing a medical device quality management system using a 292-item sampled checklist, LLMES achieved an F1 score of 0.895 on core datasets, significantly outperforming the 0.696 score of direct auditing. Additionally, LLMES attained an F1 score of 0.876 ± 0.034 across full datasets and 0.905 on a complete 1696-item checklist. The source code and datasets are publicly available on our GitHub repository: <a href="https://github.com/lyxx3rd/LLMES">https://github.com/lyxx3rd/LLMES</a></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

LLMES: an LLMs-based expert system for quality management system audits

  • 期刊论文

  • 发表日期：

    2025年10月14日

查看全文
  • Yunhan Li,
  • Zeyang Shi,
  • Yujing Li,
  • Yuntao Chen,
  • Gengshen Wu,
  • Min Yang

摘要

Many organizations still face significant challenges in document audits, such as contract reviews, accounting audits, and compliance checks, which require substantial manpower and time. Current Artificial Intelligence solutions struggle with ambiguous requirements, unclear standards, and complex documents, limiting their scalability and effectiveness. To address these challenges, we propose a novel architecture, LLMES, designed to generate a concise and independent audit checklist. This approach allows Large Language Models to perform audits and produce results without prior fine-tuning. LLMES effectively integrates complex legal documents, expert knowledge, and mandatory regulatory guidelines with large language modeling to enhance artificial intelligence-assisted auditing. In our experiments auditing a medical device quality management system using a 292-item sampled checklist, LLMES achieved an F1 score of 0.895 on core datasets, significantly outperforming the 0.696 score of direct auditing. Additionally, LLMES attained an F1 score of 0.876 ± 0.034 across full datasets and 0.905 on a complete 1696-item checklist. The source code and datasets are publicly available on our GitHub repository: https://github.com/lyxx3rd/LLMES