User Experience with ChatGPT: Insights from a Comprehensive Evaluation

Since ChatGPT was presented by OpenAI, researchers, practitioners, and curious people worldwide have rushed to experiment with its impressive capabilities. In this paper, we present our research activities aimed at carrying out a critical and in-depth analysis of ChatGPT, highlighting its enormous potential and its possible limitations. Specifically, we evaluated the user experience with ChatGPT from different points of view, from its ability to adapt to the user’s lexicon to the depth of its specific/general knowledge, from its ability to provide suggestions also in the psychological field to the efficiency with which it offers satisfactory answers compared to traditional search engines. To carry out this analysis, we administered a questionnaire to a heterogeneous sample of users: experts in specific sectors and ordinary people without particular knowledge. Statistical tests confirmed the significance of the results obtained. These results allowed us to identify some of the strengths and weaknesses of this model.