Scholarlink

<p>Monitoring just-in-time adaptive interventions (JITAIs) is important both during trialing and when the intervention is deployed in a broader healthcare program. While there is increasing interest in using artificial intelligence (AI) algorithms in JITAIs, these algorithms introduce additional complexity that requires additional monitoring. In this paper, we provide guidelines for monitoring online AI decision-making algorithms. Our guidelines include: (1) identifying potential issues, categorizing them by severity (red, yellow, and green), and (2) developing fallback methods (pre-specified procedures that are executed when an issue occurs). To make ideas concrete, we discuss algorithm monitoring systems in two case studies. In both, the monitoring systems detected real-time issues, and fallback methods both safeguarded participants and ensured quality data for post-deployment data analysis to further refine the JITAI. These guidelines and findings give teams the confidence to include online AI decision-making algorithms in JITAIs.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Effective monitoring of online AI decision-making algorithms in just-in-time adaptive interventions

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年5月4日

查看全文
  • Anna L. Trella,
  • Susobhan Ghosh,
  • Erin E. Bonar,
  • Lara Coughlin,
  • Finale Doshi-Velez,
  • Yongi Guo,
  • Pei-Yao Hung,
  • Inbal Nahum-Shani,
  • Vivek Shetty,
  • Maureen Walton,
  • Iris Yan,
  • Kelly W. Zhang,
  • Susan A. Murphy

摘要

Monitoring just-in-time adaptive interventions (JITAIs) is important both during trialing and when the intervention is deployed in a broader healthcare program. While there is increasing interest in using artificial intelligence (AI) algorithms in JITAIs, these algorithms introduce additional complexity that requires additional monitoring. In this paper, we provide guidelines for monitoring online AI decision-making algorithms. Our guidelines include: (1) identifying potential issues, categorizing them by severity (red, yellow, and green), and (2) developing fallback methods (pre-specified procedures that are executed when an issue occurs). To make ideas concrete, we discuss algorithm monitoring systems in two case studies. In both, the monitoring systems detected real-time issues, and fallback methods both safeguarded participants and ensured quality data for post-deployment data analysis to further refine the JITAI. These guidelines and findings give teams the confidence to include online AI decision-making algorithms in JITAIs.