DreaMR: The Effects of Multisensory Design on Cross-Modal Perception Across Genres with Mixed Reality Concert
摘要
Although Virtual Reality technology offers a variety of experiences, Mixed Reality (MR) is a relatively new field, offering the ability to merge real and digital environments in sophisticated ways. Perception is at the core of this integration and our experience of them: As humans, we perceive reality with our senses; we feel rain on our skin, we see the drops, we hear the sound, we can taste and smell the freshness of the air in real life. When creating and experiencing digitally enhanced realities, we still rely on the senses, but practically all VR/MR use only the audio-visual source, because other sensory modalities (e.g., tactile and olfactory) are not generally supported by the platforms. In this paper, we are concerned with the following questions: How can we reach other senses through a MR audio-visual environment? Can we create dreamlike experiences and feel music and art as our reality? Music and art therapies are well-established practices, so can MR have the potential to provide mindfulness? The pilot study presented in this paper offers insights into the nature of cross-modal associations and synesthesia interaction of senses theory as a framework for multisensory design. The benefits of multisensory stimulation are not fully explored in the field of neuroscience, but there are statements linking synesthesia (multisensory or cross-modal perception) with creativity and increased mental ability. Our results with a pilot study of our DreaMR series of experiences indicate a much greater ability for multisensory processing with MR in the general public than expected.