This chapter presents a quantitative study investigating the differences between human expert and one of the most popular generative model (ChatGPT-4) in answering different of water-related questions with focusing on multiple evaluation criteria including clarity, depth, and flow. However, the participants answered to the questions without knowing their source, they don’t know if the answers are from human expert or ChatGPT in order to ensure the unbiased comparison. The Wilcoxon signed-rank test was applied to analyze whether perceived differences between experts and AI-generated responses were statistically significant. The findings reveal that there is a significant difference between the human experts and ChatGPT answers in all criteria, as the human experts provided more accurate and understandable answers. These results highlight the essential role of human expertise in technical domains where knowledge accuracy is critical. As well as the potential for generative AI to complement, rather than replace the human experts. In future work, a qualitative research should be conducted to explore more the participant perceptions and the impact of their background factors such as career position, age, education, and prior experience with water-related topics.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Evaluating Human Expertise and Generative AI (ChatGPT) Responses to Questions on Water Challenges: A Quantitative Study

  • Suha Khalil Assayed,
  • Almoayied Assayed

摘要

This chapter presents a quantitative study investigating the differences between human expert and one of the most popular generative model (ChatGPT-4) in answering different of water-related questions with focusing on multiple evaluation criteria including clarity, depth, and flow. However, the participants answered to the questions without knowing their source, they don’t know if the answers are from human expert or ChatGPT in order to ensure the unbiased comparison. The Wilcoxon signed-rank test was applied to analyze whether perceived differences between experts and AI-generated responses were statistically significant. The findings reveal that there is a significant difference between the human experts and ChatGPT answers in all criteria, as the human experts provided more accurate and understandable answers. These results highlight the essential role of human expertise in technical domains where knowledge accuracy is critical. As well as the potential for generative AI to complement, rather than replace the human experts. In future work, a qualitative research should be conducted to explore more the participant perceptions and the impact of their background factors such as career position, age, education, and prior experience with water-related topics.