Scholarlink

Background <p>Asbestosis, a rare pneumoconiosis marked by diffuse pulmonary fibrosis, arises from prolonged asbestos exposure. Its diagnosis, guided by the Helsinki criteria, relies on exposure history, clinical findings, radiology, and lung function. However, interobserver variability complicates diagnoses and financial compensation. This study prospectively validated the sensitivity of an AI-driven assessment for asbestosis compensation in the Netherlands. Secondary objectives included evaluating specificity, accuracy, predictive values, area under the curve of the receiver operating characteristic (ROC-AUC), area under the precision-recall curve (PR-AUC), and interobserver variability.</p> Materials and methods <p>Between September 2020 and July 2022, 92 adult compensation applicants were assessed using both AI models and pulmonologists’ reviews based on Dutch Health Council criteria. The AI model assigned an asbestosis probability score: negative (&lt; 35), uncertain (35–66), or positive (≥ 66). Uncertain cases underwent additional reviews for a final determination.</p> Results <p>The AI assessment demonstrated sensitivity of 0.86 (95% confidence interval: 0.77–0.95), specificity of 0.85 (0.76–0.97), accuracy of 0.87 (0.79–0.93), ROC-AUC of 0.92 (0.84–0.97), and PR-AUC of 0.95 (0.89–0.99). Despite strong metrics, the sensitivity target of 98% was unmet. Pulmonologist reviews showed moderate to substantial interobserver variability.</p> Conclusion <p>The AI-driven approach demonstrated robust accuracy but insufficient sensitivity for validation. Addressing interobserver variability and incorporating objective fibrosis measurements could enhance future reliability in clinical and compensation settings.</p> Relevance statement <p>The AI-driven assessment for financial compensation of asbestosis showed adequate accuracy but did not meet the required sensitivity for validation.</p> Key Points <p><UnorderedList Mark="Bullet"> <ItemContent> <p>We prospectively assessed the sensitivity of an AI-driven assessment procedure for financial compensation of asbestosis.</p> </ItemContent> <ItemContent> <p>The AI-driven asbestosis probability score underperformed across all metrics compared to internal testing.</p> </ItemContent> <ItemContent> <p>The AI-driven assessment procedure achieved a sensitivity of 0.86 (95% confidence interval: 0.77–0.95). It did not meet the predefined sensitivity target.</p> </ItemContent> </UnorderedList></p> Graphical Abstract <p></p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Prospective validation of an artificial intelligence assessment in a cohort of applicants seeking financial compensation for asbestosis (PROSBEST)

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2025年8月15日

查看全文
  • Illaa Smesseim,
  • Kevin B. W. Groot Lipman,
  • Stefano Trebeschi,
  • Martijn M. Stuiver,
  • Renaud Tissier,
  • Jacobus A. Burgers,
  • Cornedine J. de Gooijer

摘要

Background

Asbestosis, a rare pneumoconiosis marked by diffuse pulmonary fibrosis, arises from prolonged asbestos exposure. Its diagnosis, guided by the Helsinki criteria, relies on exposure history, clinical findings, radiology, and lung function. However, interobserver variability complicates diagnoses and financial compensation. This study prospectively validated the sensitivity of an AI-driven assessment for asbestosis compensation in the Netherlands. Secondary objectives included evaluating specificity, accuracy, predictive values, area under the curve of the receiver operating characteristic (ROC-AUC), area under the precision-recall curve (PR-AUC), and interobserver variability.

Materials and methods

Between September 2020 and July 2022, 92 adult compensation applicants were assessed using both AI models and pulmonologists’ reviews based on Dutch Health Council criteria. The AI model assigned an asbestosis probability score: negative (< 35), uncertain (35–66), or positive (≥ 66). Uncertain cases underwent additional reviews for a final determination.

Results

The AI assessment demonstrated sensitivity of 0.86 (95% confidence interval: 0.77–0.95), specificity of 0.85 (0.76–0.97), accuracy of 0.87 (0.79–0.93), ROC-AUC of 0.92 (0.84–0.97), and PR-AUC of 0.95 (0.89–0.99). Despite strong metrics, the sensitivity target of 98% was unmet. Pulmonologist reviews showed moderate to substantial interobserver variability.

Conclusion

The AI-driven approach demonstrated robust accuracy but insufficient sensitivity for validation. Addressing interobserver variability and incorporating objective fibrosis measurements could enhance future reliability in clinical and compensation settings.

Relevance statement

The AI-driven assessment for financial compensation of asbestosis showed adequate accuracy but did not meet the required sensitivity for validation.

Key Points

We prospectively assessed the sensitivity of an AI-driven assessment procedure for financial compensation of asbestosis.

The AI-driven asbestosis probability score underperformed across all metrics compared to internal testing.

The AI-driven assessment procedure achieved a sensitivity of 0.86 (95% confidence interval: 0.77–0.95). It did not meet the predefined sensitivity target.

Graphical Abstract