Voice Commands with Virtual Assistant in Mixed Reality Telepresence

Mixed Reality (MR) telepresence is an expanding field in marketing research and practice especially in computer vision due to its ability to connect people in a remote location. As voice command and speech recognition technology have also been significant milestones in the twenty-first century, this technology has the advantage to be used in MR telepresence for interaction. Meanwhile, the virtual assistant avatar’s implementation in MR telepresence can provide support for the interaction by reducing the workload and improving the performance of the interaction in collaborative MR telepresence. Therefore, in this paper, we propose to explore the voice commands interaction and describe the implementation with the virtual assistant avatar in MR telepresence. We implement speech-to-text (STT) conversion and utilize the semantics of Natural Language Processing (NLP) for the voice command. Meanwhile, for the avatar to respond, the speech service is used to convert the text-to-speech (TTS) input. This paper ends with a conclusion, limitations and suggestions for future works.