Perception-Driven Design Approach: Towards Interaction Design for Simulating and Evoking Tactile Properties via Digital Interfaces

Nowadays, digital and virtual user experiences lack of multisensory feedback necessary to assess product and material properties. Scientific literature related to haptic interface technology to address this issue is still limited and mostly employs a simulation-type approach. Yet, there are a few studies using and combining sensory modalities, such as visual and auditory stimuli. On the one hand, this research provides a literature framework with respect to interaction with digital and virtual materials, analyzing relevant case studies and providing guidelines for designing the interaction. On the other hand, we conceptualize a cross-modal approach, i.e., Perception-Driven Design, by exploiting two human senses - vision and hearing - in order to provide HCI researchers and interaction designers a novel approach not only to “simulate” but also “evoke” the actual tactile feedback related to the material properties. These interfaces would allow users to interact with displayed material subjected to deformation based on the gesture the user performs and producing visual feedback. Additionally, audio feedback could be recorded or synthetized in real-time, by exploiting simulated sounds of real materials or evocative sounds, in combination to the visual material deformation. The research objective is to develop a design approach for future researchers in the field of interaction, useful for designing digital and virtual interfaces valuable in all fields that highly rely on tactile perception as their pillar (e. g., fashion, furniture, automotive).