Scholarlink

<p>The performance of large language models (LLMs)-DeepSeek in the generation of in-training examination questions in radiology resident education is understudied. In this study, multiple-choice questions (MCQs) were generated by DeepSeek and written by radiology experts according to Chinese educational standards. Fourteen MCQs (A1, A2, A3/A4 types) from each source were randomly intermixed into a 28-item online test, administered to 40 radiology residents (17 second-year, 23 third-year residents). Participants completed the closed-book exam by answering each question, identifying its origin (DeepSeek or expert), and rating it on perceived difficulty, curriculum relevance, overall quality, and clinical realism using standardized scales. There were no significant differences between DeepSeek-generated and expert-written questions in overall correct response rates or source attribution accuracy. Furthermore, item-type stratification revealed that residents had similar correct response rates for DeepSeek-generated and expert-written A1 questions, while lower correct response rates for DeepSeek-generated A3/A4 questions than expert-written ones. Subjective evaluations showed lower ratings of clinical scenario realism for DeepSeek-generated A2 questions compared to expert-written ones, with a more pronounced difference among third-year residents. DeepSeek demonstrates potential in the generation of in-training examination questions in radiology resident education. However, it is less effective than human experts in creating higher-order questions requiring clinical scenarios.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Performance of DeepSeek in the generation of in-training examination questions in radiology resident education

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月24日

查看全文
  • Wei Qian,
  • Kaicheng Li,
  • Fang Cao,
  • Shuyue Wang,
  • Jin Mao,
  • Chao Wang

摘要

The performance of large language models (LLMs)-DeepSeek in the generation of in-training examination questions in radiology resident education is understudied. In this study, multiple-choice questions (MCQs) were generated by DeepSeek and written by radiology experts according to Chinese educational standards. Fourteen MCQs (A1, A2, A3/A4 types) from each source were randomly intermixed into a 28-item online test, administered to 40 radiology residents (17 second-year, 23 third-year residents). Participants completed the closed-book exam by answering each question, identifying its origin (DeepSeek or expert), and rating it on perceived difficulty, curriculum relevance, overall quality, and clinical realism using standardized scales. There were no significant differences between DeepSeek-generated and expert-written questions in overall correct response rates or source attribution accuracy. Furthermore, item-type stratification revealed that residents had similar correct response rates for DeepSeek-generated and expert-written A1 questions, while lower correct response rates for DeepSeek-generated A3/A4 questions than expert-written ones. Subjective evaluations showed lower ratings of clinical scenario realism for DeepSeek-generated A2 questions compared to expert-written ones, with a more pronounced difference among third-year residents. DeepSeek demonstrates potential in the generation of in-training examination questions in radiology resident education. However, it is less effective than human experts in creating higher-order questions requiring clinical scenarios.