A Relative Pitch Based Approach to Non-verbal Vocal Interaction as a Continuous and One-Dimensional Controller

Non-verbal vocal interaction (NVVI) allows for non-speech vocal inputs to be used to control applications. Prior work in the niche field has explored a variety of modalities of NVVI such as whistling, humming, blowing, and tongue clicking. Different techniques, including pitch gestures, continuous, and discrete control have been explored. In this work, we analyze results from a large-scale study exploring a pitch-based NVVI modality that uses a relative pitch interaction technique to offer a continuous mode of one-dimensional control. A user study is outlined and was performed via Amazon Mechanical Turk for recruitment and study access, and a modularized study website where participants controlled an HTML slider with the NVVI technique by humming and whistling. Additionally, participants performed similar tasks using the computer mouse as a baseline. In total, 72 participants’ results are considered for analysis. Results show that the pitch based NVVI technique used in this study does not follow Fitts’ Law, participants did not perform as well with the NVVI technique as they did with the computer mouse, and that participants experienced a significantly higher task workload using the NVVI technique than with the computer mouse.