Uncertainty-aware abstention in medical diagnosis based on medical texts
摘要
This study addresses the reliability of AI-assisted medical diagnosis through selective prediction, where models can abstain from decisions based on predictive uncertainty. Such selective prediction (or abstention) approaches are usually based on modeling the predictive uncertainty of the machine learning models involved. We systematically evaluate uncertainty quantification methods for medical text analysis across heterogeneous tasks and datasets, including binary mortality prediction from discharge summaries in MIMIC-III, multi-label ICD-10 code assignment in MIMIC-IV, multi-class diagnosis prediction from a private outpatient corpus, and mental health detection of depression and anxiety from essays, social media posts, and clinical narratives. In addition to comparing uncertainty methods, we propose HUQ-2, a new extension of the hybrid uncertainty quantification method that effectively combines aleatoric and epistemic uncertainty to support reliable selective prediction. Experiments demonstrate the effectiveness of HUQ-2 in capturing and evaluating uncertainty, paving the way for more reliable and interpretable applications in medical text analysis. For multi-label code prediction, we further introduce label-level rejection, allowing the model to abstain on individual codes rather than entire cases, which yields substantial gains in selective prediction performance and highlights a promising direction for safer medical NLP systems.