A Case Study in XR Live Performance

This paper discusses the design of a system for Extended Realities (XR) in live arts performances. XR, a broad concept and medium, enhances the capabilities of Virtual and Augmented Reality (VR/AR) by incorporating spatial audio and multiple projections for audiences. In live performance scenarios, XR has the potential to define a novel, specific type of audience engagement, which is largely unexplored. This paper specifically examines the modular structure of the system, prioritizing a high-level description over detailed implementation specifics. However, details are included as needed for clarity and to provide context within the XR medium. According to the design, a VR/AR performer wearing a headset can interact with virtual objects, controlling the audio and visuals generated during the performance. The system is divided into modules dedicated to specific functions: audio generation, video generation, event scheduling, parameter mapping, and inter-machine connections (typically, the system requires several machines to run in parallel due to performance and input/output necessities). These modules are overseen by a main module, the VR/AR performer module. The system is designed to explore and leverage specific affordances of XR as a medium. It envisions XR performance as a stratified experience of multiple realities, such as the simultaneous existence of VR/AR and screen-based content. Moreover, it ensures that these different types of content are connected and interdependent, a feature crucial for delivering immersive experiences to audiences who cannot directly experience VR/AR content. To achieve this, a dynamic ecosystem of devices is supported to accommodate various perspectives and dimensions of reality. The system has been entirely designed to cater to these needs and facilitate rapid content creation and prototyping.