Study of natural language communication with avatar in virtual space

Artificial reality, stemming from telexistence, is evolving into the virtual spaces, emphasizing user interaction across language, multimodal, multitask, and embedded interactions, as well as highly accurate virtual reality (VR) environments and applications. Furthermore, human–agent interaction via avatars requires knowledge implementation. Recently, it has become possible to utilize large-scale language models, and development is advancing. In this study, we implemented sentence-transformers by optimizing parameters and performing instructional learning to respond accordingly to specific keywords, enhancing the quality of natural language conversation. A virtual space in conjunction with the precision computer graphics (CG) of our university's color research center was constructed, and verified its effectiveness through experiential experiments. From this experiment, it was found that the accuracy of the voice recognition was inadequate and the reliability of the avatar was rated low. Therefore, improvements need to be made to enhance the accuracy of the voice recognition. We would also like to increase the number of training languages in order to facilitate smoother conversations.