Scholarlink

<p>Decision support pipelines increasingly combine machine learning predictions with human judgment, yet most public benchmarks evaluate model outputs only and do not encode the interaction process that determines final decisions. This limits reproducible analysis of when human intervention improves or degrades system-level performance. We introduce HCCD-DS v2, a transparent and configurable synthetic benchmark dataset that models decision-time interaction between an AI recommender and simulated users with continuous expertise levels under contextual uncertainty. Each decision instance links input context, system confidence, explainability metadata, human acceptance/override behavior, override rationale, and realized outcome. The dataset simulates distinct profiles for healthcare, cybersecurity, and Internet of Things (IoT) scenarios to support cross-domain evaluation. Empirical analyses and benchmark experiments show that human intervention is condition-dependent: expert overrides are more likely to improve outcomes, while novice overrides more often reduce success. By exposing this interaction in a unified and reusable resource under explicit behavioral assumptions (sequential trust updates, individual risk tolerance, and explanation sensitivity), HCCD-DS v2 provides a controlled environment for benchmarking trust calibration, explanation-aware interaction policies, and algorithms that learn to defer. We discuss the limits of predictive recoverability in simulated human behavior and frame the benchmark’s F1-score transitions as a measure of policy complexity.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

HCCD-DS v2: a transparent synthetic benchmark for human–AI decision support under contextual uncertainty

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月24日

查看全文
  • Kadir Kesgin

摘要

Decision support pipelines increasingly combine machine learning predictions with human judgment, yet most public benchmarks evaluate model outputs only and do not encode the interaction process that determines final decisions. This limits reproducible analysis of when human intervention improves or degrades system-level performance. We introduce HCCD-DS v2, a transparent and configurable synthetic benchmark dataset that models decision-time interaction between an AI recommender and simulated users with continuous expertise levels under contextual uncertainty. Each decision instance links input context, system confidence, explainability metadata, human acceptance/override behavior, override rationale, and realized outcome. The dataset simulates distinct profiles for healthcare, cybersecurity, and Internet of Things (IoT) scenarios to support cross-domain evaluation. Empirical analyses and benchmark experiments show that human intervention is condition-dependent: expert overrides are more likely to improve outcomes, while novice overrides more often reduce success. By exposing this interaction in a unified and reusable resource under explicit behavioral assumptions (sequential trust updates, individual risk tolerance, and explanation sensitivity), HCCD-DS v2 provides a controlled environment for benchmarking trust calibration, explanation-aware interaction policies, and algorithms that learn to defer. We discuss the limits of predictive recoverability in simulated human behavior and frame the benchmark’s F1-score transitions as a measure of policy complexity.