Text entry is an important task in virtual reality (VR), and most existing methods require hand involvement, while hands-free typing has great potential for applications in mobile scenarios. Existing hands-free text entry methods are usually implemented by combining the head and eyes with techniques such as Dwell, Blink and Gesture, which can easily fatigue the user. In this paper, we propose LipText, a lip-tracking-based text entry method in VR. We use a neural network to perform letter-level prediction on the lip data captured by the facial tracker and use head-based selection as an auxiliary to improve the accuracy. We conduct a user study to evaluate our method, the results show a typing speed of 8.63 WPM for the novice group, 9.81 WPM for the potential expert group, and the highest recorded typing speed is 11.13 WPM achieved by a potential expert. Our method is also novice-friendly, and their typing speed increased by 64.38% over a six-day practice.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

LipText: Lip Tracking Based Text Entry in VR

  • Jiaye Leng,
  • Zijun Wang,
  • Jian Wu,
  • Lili Wang

摘要

Text entry is an important task in virtual reality (VR), and most existing methods require hand involvement, while hands-free typing has great potential for applications in mobile scenarios. Existing hands-free text entry methods are usually implemented by combining the head and eyes with techniques such as Dwell, Blink and Gesture, which can easily fatigue the user. In this paper, we propose LipText, a lip-tracking-based text entry method in VR. We use a neural network to perform letter-level prediction on the lip data captured by the facial tracker and use head-based selection as an auxiliary to improve the accuracy. We conduct a user study to evaluate our method, the results show a typing speed of 8.63 WPM for the novice group, 9.81 WPM for the potential expert group, and the highest recorded typing speed is 11.13 WPM achieved by a potential expert. Our method is also novice-friendly, and their typing speed increased by 64.38% over a six-day practice.