Scholarlink

This paper introduces a real-time multimodal AI framework that integrates YOLO-based object detection with speech recognition for intuitive, human-like interaction. Utilizing OpenCV and the Python SpeechRecognition library, the system supports hands-free control and automated YOLO-format dataset generation. Achieving 91% mAP in object detection and 96% speech accuracy, it operates at 25 FPS on CPU hardware. The framework remains robust under low-light (85%), occlusion (78%), and fast motion (82%) conditions, demonstrating suitability for robotics, assistive technologies, and surveillance. Future enhancements include multi-object detection, multilingual input, and GPU acceleration.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An Integrated Framework for Real-Time Object Detection and Speech-Driven Interaction: Advancing Multimodal Human-Like Intelligence

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Dev Patel,
  • Davinder Paul Singh,
  • Pawandeep Kour,
  • Prachi Chhabra,
  • Debabrata Swain,
  • Shubham Mahajan,
  • Manish Kumar,
  • Nikhith Vasa

摘要

This paper introduces a real-time multimodal AI framework that integrates YOLO-based object detection with speech recognition for intuitive, human-like interaction. Utilizing OpenCV and the Python SpeechRecognition library, the system supports hands-free control and automated YOLO-format dataset generation. Achieving 91% mAP in object detection and 96% speech accuracy, it operates at 25 FPS on CPU hardware. The framework remains robust under low-light (85%), occlusion (78%), and fast motion (82%) conditions, demonstrating suitability for robotics, assistive technologies, and surveillance. Future enhancements include multi-object detection, multilingual input, and GPU acceleration.