Development of a Human-Agent Dialogue Interaction System Using Comic Symbols for Enriching Agent Expression
摘要
Comic symbols that express character states and emotions in comics are intuitively easy to understand, and they can be adapted to not only human-like characters but also animals and objects. Previous studies show the usefulness of comic symbols in human-agent interaction. However, these studies only display predetermined or limited types of comic symbols. In this paper, we describe the development of a system that dynamically utilizes comic symbols according to the content of dialogues. For the first step of the research, we focused on human-like characters and annotated a subset of a comic dataset (Manga109) by identifying the types, display positions, and the number of comic symbols. The annotated data were utilized to fine-tune a large language model. With this fine-tuned model, we developed a system capable of determining the appropriate type, position, and quantity of comic symbols based on the conversational context and spoken dialogue. Leveraging this model, we developed an interactive system in which an on-screen 3D agent engages in dialogues effectively using comic symbols. For future research, we discuss the potential application of the usage of comic symbols in human-robot interaction. We plan to develop a system in which robots can communicate emotions using comic symbols by utilizing mixed reality technology.