Evaluating Human Expertise and Generative AI (ChatGPT) Responses to Questions on Water Challenges: A Quantitative Study
摘要
This chapter presents a quantitative study investigating the differences between human expert and one of the most popular generative model (ChatGPT-4) in answering different of water-related questions with focusing on multiple evaluation criteria including clarity, depth, and flow. However, the participants answered to the questions without knowing their source, they don’t know if the answers are from human expert or ChatGPT in order to ensure the unbiased comparison. The Wilcoxon signed-rank test was applied to analyze whether perceived differences between experts and AI-generated responses were statistically significant. The findings reveal that there is a significant difference between the human experts and ChatGPT answers in all criteria, as the human experts provided more accurate and understandable answers. These results highlight the essential role of human expertise in technical domains where knowledge accuracy is critical. As well as the potential for generative AI to complement, rather than replace the human experts. In future work, a qualitative research should be conducted to explore more the participant perceptions and the impact of their background factors such as career position, age, education, and prior experience with water-related topics.