A cross-texture haptic model based on tactile feature fusion

Virtual haptic textures can enhance the realism of human–computer interaction, enabling the users in multimedia environments to not only see the virtual objects but also touch the texture details of the objects’ surface. Data-driven approaches have been widely used in haptic texture modeling, but the existing models are either only able to model one texture at a time, or adopt a multimodal approach which increases the modeling complexity. This article presents a method for implementing a cross-texture haptic model capable of generating tactile vibration according to the given texture type and user’s motion. The modeling method is based on tactile feature fusion, primarily divided into two steps. First, hand-crafted texture features, motion features and vibration features are extracted from the pre-collected raw tactile data. These features are interpretable and have their own physical meanings. Then, a neural network with fully connected layers is devised for its non-linear advantage in simulating vibration response and trained to fuse texture feature and motion feature before mapping them to vibration feature. The proposed model is validated through objective and subjective experiments using two datasets. The experimental results demonstrate its accuracy in generating numerical signals and its capability to run in real-time.