The research paper delivers an innovative concept of gestural intelligence based on virtual mouse technology that employs modern artificial intelligence techniques, CNN (Convolutional Neural Networks), and NLP (Natural Language Processing). The system features a touchless interface designed to replace traditional human-computer interaction methods. MediaPipe and pybind11 are utilized for real-time gesture tracking using a standard webcam, enabling actions like cursor movement, clicking, scrolling, and dragging without physical contact. The solution reduces costs and enhances accessibility by eliminating the need for additional hardware. The interface is highly adaptive, making it suitable for both general and specialized applications. Experimental results show strong performance, with 94% accuracy in both gesture and voice command recognition under optimal conditions. The system operates efficiently with low latency and minimal resource usage, confirming its effectiveness and reliability across various environments in modern HCI applications.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Integrating Hand Gesture Recognition with Voice Assistants: Enhancing Human-Computer Interaction Through Multimodal Intelligence

  • K. A. Priyanka,
  • S. Aishwarya,
  • J. Jayanth,
  • M. Suriya

摘要

The research paper delivers an innovative concept of gestural intelligence based on virtual mouse technology that employs modern artificial intelligence techniques, CNN (Convolutional Neural Networks), and NLP (Natural Language Processing). The system features a touchless interface designed to replace traditional human-computer interaction methods. MediaPipe and pybind11 are utilized for real-time gesture tracking using a standard webcam, enabling actions like cursor movement, clicking, scrolling, and dragging without physical contact. The solution reduces costs and enhances accessibility by eliminating the need for additional hardware. The interface is highly adaptive, making it suitable for both general and specialized applications. Experimental results show strong performance, with 94% accuracy in both gesture and voice command recognition under optimal conditions. The system operates efficiently with low latency and minimal resource usage, confirming its effectiveness and reliability across various environments in modern HCI applications.