ExpertGen: A Comparative Analysis of User Performance, Cognitive Workload, and Trust in Domain-Tailored Generative AI
摘要
Generative artificial intelligence has transformed multiple industries, including education, healthcare, and finance, by generating text, images, video, and audio content. Despite these advancements, existing generative AI models often fail to fulfill all expectations, generating factually incorrect or misleading responses, which impairs system accountability and user trust. To overcome this limitation, we designed ExpertGen, a generative AI system tailored for specific domains, designed to provide real-time, accurate, and context-specific insights. This study evaluates the effectiveness of ExpertGen on three major dimensions: learning performance, cognitive workload, and user trust. Experimental results show that ExpertGen significantly improves learning outcomes, with an average performance of 21.08 points higher than traditional e-books, and outperforming ChatGPT in domain-specific tasks. In addition, NASA-TLX cognitive workload analysis proves that ExpertGen significantly reduces the cognitive demands of e-books and its usability is comparable to ChatGPT. Trust analysis shows that users trust ExpertGen more than ChatGPT. These studies suggest that generative AI systems need to enhance reliability, transparency, and domain adaptability to improve user acceptance and educational outcomes.