User Experience Evaluation Methods in Mixed Reality Environments

Interest in the metaverse and telepresence has sparked technological advancements, enabling us to extend our perceived reality with off-the-shelf devices. Particularly, we shall be focusing on mixed reality (MR) which consists in merging the real and the virtual world through environment synchronization (ES). In this interaction paradigm, a challenge arises for capturing and understanding what users think and how they physically/emotionally react when immersed in MR. Our theoretical approach stems from user experience (UX) research, by evaluating aspects such as usability, utility, and satisfaction of use. However, we hypothesize that MR also involves specific factors unique to this interface. Therefore, in this literature review we shall be looking at three questions: (1) what methods have been used to evaluate UX in MR? (2) what aspects have been assessed in UX evaluation? And (3) what devices have been used in these evaluations? We consulted 3 scientific databases and our search yielded 737 results from the last 10 years. Results show a growing trend on research interest from 2019 onwards. Secondly, we found that studies most frequently rely on what the users respond in post-experiment questionnaires and interviews, neglecting other mid-experiment methods such as Think Aloud, or physiological signals sensing. Furthermore, usability, enjoyment and simulator sickness remain the most recurrent factors assessed, but only two studies included the quality assessment of ES, and security/privacy aspects. Other specific UX aspects in MR include perceived immersion, navigation, and device ergonomics. We note a lack of a well-documented UX model particularized to incorporate all these specific aspects for comprehensive evaluation. To the best of our knowledge expert judgment remains the standard on which aspects assess in MR environments. The contribution of this paper is a snapshot of how UX evaluation has been conducted in recent literature. We provide a list of methods and instruments for designers and developers to incorporate users’ insights for improvement. For future work we intend to expand this research to a systematic literature review to overcome the limitations of this work and supply evidence for proposing a UX model for MR environments.