Scholarlink

<p>Artificial intelligence is revolutionizing scientific research, yet its growing integration into laboratory environments presents critical safety challenges. Large language models and vision language models now assist in experiment design and procedural guidance, yet their ‘illusion of understanding’ may lead researchers to overtrust unsafe outputs. Here we show that current models remain far from meeting the reliability needed for safe laboratory operation. We introduce LabSafety Bench, a comprehensive benchmark that evaluates models on hazard identification, risk assessment and consequence prediction across 765 multiple-choice questions and 404 realistic laboratory scenarios, encompassing 3,128 open-ended tasks. Evaluations on 19 advanced large language models and vision language models show that no model evaluated on hazard identification surpasses 70% accuracy. While proprietary models perform well on structured assessments, they do not show a clear advantage in open-ended reasoning. These results underscore the urgent need for specialized safety evaluation frameworks before deploying artificial intelligence systems in real laboratory settings.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Benchmarking large language models on safety risks in scientific laboratories

  • 期刊论文

  • 发表日期：

    2026年1月14日

查看全文
  • Yujun Zhou,
  • Jingdong Yang,
  • Yue Huang,
  • Kehan Guo,
  • Zoe Emory,
  • Bikram Ghosh,
  • Amita Bedar,
  • Sujay Shekar,
  • Zhenwen Liang,
  • Pin-Yu Chen,
  • Tian Gao,
  • Werner Geyer,
  • Nuno Moniz,
  • Nitesh V. Chawla,
  • Xiangliang Zhang

摘要

Artificial intelligence is revolutionizing scientific research, yet its growing integration into laboratory environments presents critical safety challenges. Large language models and vision language models now assist in experiment design and procedural guidance, yet their ‘illusion of understanding’ may lead researchers to overtrust unsafe outputs. Here we show that current models remain far from meeting the reliability needed for safe laboratory operation. We introduce LabSafety Bench, a comprehensive benchmark that evaluates models on hazard identification, risk assessment and consequence prediction across 765 multiple-choice questions and 404 realistic laboratory scenarios, encompassing 3,128 open-ended tasks. Evaluations on 19 advanced large language models and vision language models show that no model evaluated on hazard identification surpasses 70% accuracy. While proprietary models perform well on structured assessments, they do not show a clear advantage in open-ended reasoning. These results underscore the urgent need for specialized safety evaluation frameworks before deploying artificial intelligence systems in real laboratory settings.