Musical Syntax and Sonification of Voice and Speech Interfaces: A Case Study in Turn-Taking

Voice and speech (V&S) technologies have been applied to user interface design for many years, with mixed results regarding standards long established in the design community by human–computer interaction (HCI) professionals. The minimum standard is to create user interfaces that are effective, efficient, and satisfying. In this work, I will discuss why this standard is rarely met and how we might use additional interface channels to supplement the core capabilities provided by automatic speech recognition (ASR) technology. These channels consist of ancillary sound media, including functional musical syntax and non-musical earcons. A specific case study—recently developed and tested—is presented in some detail.