Scholarlink

Background <p>Accurate preoperative risk identification is critical for patient safety and postoperative outcomes. Anaesthesiologists make decisions on the basis of ASA classification and additional parameters. Artificial intelligence (AI)-based decision support may offer more objective judgments.</p> Methods <p>In this retrospective multi-rater study, four anaesthesiologists and an AI system independently evaluated 1,000 cases. ASA class, postoperative ICU requirement, anaesthesia preference, intraoperative risk prediction, and additional recommendations were assessed. Concordance was analysed using Krippendorff’s alpha, Cohen’s kappa, Gwet’s AC2, and PABAK, with percentage agreement estimated by bootstrapping. AI–physician agreement was further examined using fixed-effects logistic regression including clinician sex and professional experience as covariates.</p> Results <p>Physician–physician agreement was generally good to excellent across outcomes, whereas physician–AI agreement was lower and variable when assessed using κ, PABAK, Gwet’s AC2, and observed agreement (Pₒ). The highest AI concordance was observed for intraoperative risk prediction and ICU requirement, while the lowest was for anaesthesia preference. Exploratory analyses suggested that AI–physician concordance may vary by clinician experience and sex; no significant effects of sex or experience were observed for intraoperative anaesthesia-related risk prediction.</p> Conclusion <p>Although AI shows high concordance with physician decisions in objective/algorithmic domains, concordance remains limited in contextual and experience-based domains (anaesthesia preference). The findings support positioning AI as a safe ‘second eye/warning’ tool within human-in-the-loop workflows, rather than as an independent authority. Prospective, externally validated studies are needed.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Comparison of human–AI agreement in ASA scoring by gender and duration of clinical experience: a real-world study

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年2月18日

查看全文
  • Tuba Çatak,
  • Ahmet Aksu,
  • Ali Özgül Saltalı,
  • Büşra Berilgen

摘要

Background

Accurate preoperative risk identification is critical for patient safety and postoperative outcomes. Anaesthesiologists make decisions on the basis of ASA classification and additional parameters. Artificial intelligence (AI)-based decision support may offer more objective judgments.

Methods

In this retrospective multi-rater study, four anaesthesiologists and an AI system independently evaluated 1,000 cases. ASA class, postoperative ICU requirement, anaesthesia preference, intraoperative risk prediction, and additional recommendations were assessed. Concordance was analysed using Krippendorff’s alpha, Cohen’s kappa, Gwet’s AC2, and PABAK, with percentage agreement estimated by bootstrapping. AI–physician agreement was further examined using fixed-effects logistic regression including clinician sex and professional experience as covariates.

Results

Physician–physician agreement was generally good to excellent across outcomes, whereas physician–AI agreement was lower and variable when assessed using κ, PABAK, Gwet’s AC2, and observed agreement (Pₒ). The highest AI concordance was observed for intraoperative risk prediction and ICU requirement, while the lowest was for anaesthesia preference. Exploratory analyses suggested that AI–physician concordance may vary by clinician experience and sex; no significant effects of sex or experience were observed for intraoperative anaesthesia-related risk prediction.

Conclusion

Although AI shows high concordance with physician decisions in objective/algorithmic domains, concordance remains limited in contextual and experience-based domains (anaesthesia preference). The findings support positioning AI as a safe ‘second eye/warning’ tool within human-in-the-loop workflows, rather than as an independent authority. Prospective, externally validated studies are needed.