Understanding a Symphony Orchestra by Reexperience in Virtual Reality

The aim of the project Symotiv ( http://symotiv.de ) is to introduce people to classical music through interaction with a virtual orchestra. With the help of Virtual Reality, it does not only enable to immerse oneself in a three-dimensional, virtual concert hall, but also to experience images and sound from the perspective of a musician. This opened the possibility of experiencing and understanding what is happening from new spatial, but above all sonic perspectives. Therefore, we developed a workflow to capture the motions and sound of 50 musicians and the conductor by using a camera-based Machine Learning approach.