Chatbot Technology Assessment: 40 Cases from Greece

In recent years, the field of Artificial Intelligence has seen significant progress, particularly in the development of chatbots via Natural Language Processing (NLP) technology. Recently, however, there has been a real race in this sector with major technology companies constantly presenting new improved solutions. However, the Greek reality presents several peculiarities and difficulties in adopting modern solutions, both due to the idiosyncrasies and rarity of the language and the limited funding capabilities of the Greek economy. The purpose of this research is to evaluate the performance of chatbots in terms of the quality of their responses regarding relevance, naturalness, cohesion, accuracy, vocabulary, as well as to assess the user experience and satisfaction. Another goal is to gain a comprehensive comparative picture of chatbot operation in Greece, both per question and in comparison, between relevant questions. A guided interview with closed-type questions was chosen as the method of evaluation. The aim is to obtain structured and quantified data in an area where the average internet user is not fully familiarized and does not have previous relevant evaluation experience. Conclusions were drawn per question in order to evaluate the level of solutions in a focused and comparative way to identify possible trends and to confirm the consistency of the responses.