AI technologies facilitated by Large Language Models (LLMs) are increasingly used in Software Engineering, particularly to support developers in writing, understanding and testing programming code. Meanwhile, UI/UX engineers remain somehow detached from this revolutionary automation, as the cornerstone of the User-Centered Design (UCD) is involvement of real and representative users. We believe that while this approach is primarily justified, certain Human-Computer Interaction (HCI) aspects could be polished based on AI-supported tests. In our paper, we explore the capabilities of LLM-based ChatGPT service to simulate personas that represent different user groups, identified by gender, age, and education level. For this end, we ask ChatGPT in English and Russian to generate short textual associations after contextually prompting it to “be” one of the 4 specially devised persons. We further perform linguistic analysis of the 200 association units, relying on TTR (Type-Token Ratio) measure. We find that the difference in associations for different stimuli is much higher than for different personas, which suggests that ChatGPT is relatively poor at simulating them. The outcome of our study might be a step towards the development of LLM-based user behavior models (UBMs) that can facilitate testing automation in the UCD process.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Multiple Personalities by Order: Can ChatGPT Simulate Personas for User-Centered Design?

  • Maxim Bakaev,
  • Svetlana Gorovaia,
  • Olga Mitrofanova

摘要

AI technologies facilitated by Large Language Models (LLMs) are increasingly used in Software Engineering, particularly to support developers in writing, understanding and testing programming code. Meanwhile, UI/UX engineers remain somehow detached from this revolutionary automation, as the cornerstone of the User-Centered Design (UCD) is involvement of real and representative users. We believe that while this approach is primarily justified, certain Human-Computer Interaction (HCI) aspects could be polished based on AI-supported tests. In our paper, we explore the capabilities of LLM-based ChatGPT service to simulate personas that represent different user groups, identified by gender, age, and education level. For this end, we ask ChatGPT in English and Russian to generate short textual associations after contextually prompting it to “be” one of the 4 specially devised persons. We further perform linguistic analysis of the 200 association units, relying on TTR (Type-Token Ratio) measure. We find that the difference in associations for different stimuli is much higher than for different personas, which suggests that ChatGPT is relatively poor at simulating them. The outcome of our study might be a step towards the development of LLM-based user behavior models (UBMs) that can facilitate testing automation in the UCD process.