Generative artificial intelligence has transformed multiple industries, including education, healthcare, and finance, by generating text, images, video, and audio content. Despite these advancements, existing generative AI models often fail to fulfill all expectations, generating factually incorrect or misleading responses, which impairs system accountability and user trust. To overcome this limitation, we designed ExpertGen, a generative AI system tailored for specific domains, designed to provide real-time, accurate, and context-specific insights. This study evaluates the effectiveness of ExpertGen on three major dimensions: learning performance, cognitive workload, and user trust. Experimental results show that ExpertGen significantly improves learning outcomes, with an average performance of 21.08 points higher than traditional e-books, and outperforming ChatGPT in domain-specific tasks. In addition, NASA-TLX cognitive workload analysis proves that ExpertGen significantly reduces the cognitive demands of e-books and its usability is comparable to ChatGPT. Trust analysis shows that users trust ExpertGen more than ChatGPT. These studies suggest that generative AI systems need to enhance reliability, transparency, and domain adaptability to improve user acceptance and educational outcomes.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ExpertGen: A Comparative Analysis of User Performance, Cognitive Workload, and Trust in Domain-Tailored Generative AI

  • Nina Jiang,
  • Wei Zhou,
  • Sogand Hasanzadeh,
  • Vincent G. Duffy

摘要

Generative artificial intelligence has transformed multiple industries, including education, healthcare, and finance, by generating text, images, video, and audio content. Despite these advancements, existing generative AI models often fail to fulfill all expectations, generating factually incorrect or misleading responses, which impairs system accountability and user trust. To overcome this limitation, we designed ExpertGen, a generative AI system tailored for specific domains, designed to provide real-time, accurate, and context-specific insights. This study evaluates the effectiveness of ExpertGen on three major dimensions: learning performance, cognitive workload, and user trust. Experimental results show that ExpertGen significantly improves learning outcomes, with an average performance of 21.08 points higher than traditional e-books, and outperforming ChatGPT in domain-specific tasks. In addition, NASA-TLX cognitive workload analysis proves that ExpertGen significantly reduces the cognitive demands of e-books and its usability is comparable to ChatGPT. Trust analysis shows that users trust ExpertGen more than ChatGPT. These studies suggest that generative AI systems need to enhance reliability, transparency, and domain adaptability to improve user acceptance and educational outcomes.