This paper presents an auditory perception system for social robots in indoor environments, designed to detect, localize, and categorize environmental sounds in real time, allowing the robot to adapt its behavior according to the situation’s relevance. The system uses a transformer-based neural model trained on a publicly available dataset of domestic audio to recognize everyday sound events. The signal is captured by a circular four-microphone array mounted on the robot, which also estimates the direction of sound arrival, enabling the spatial mapping of acoustic events in the environment. Detected sounds are classified into three categories: environmental (ignored), supervised (requiring human attention), and emergency (requiring immediate action). The system has been implemented as a package in ROS 2, a robotic middleware framework, and validated in real domestic scenarios. Results show that integrating auditory perception allows the robot to adapt its behavior more intelligently and safely to unexpected acoustic events.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Environmental Sound Recognition for Human-Robot Interaction in Social Robots

  • Irene González Fernández,
  • David Sobrín-Hidalgo,
  • Juan Diego Peña-Narvaez,
  • Francisco J. Rodríguez Lera,
  • Camino Fernández Llamas,
  • Francisco Martín

摘要

This paper presents an auditory perception system for social robots in indoor environments, designed to detect, localize, and categorize environmental sounds in real time, allowing the robot to adapt its behavior according to the situation’s relevance. The system uses a transformer-based neural model trained on a publicly available dataset of domestic audio to recognize everyday sound events. The signal is captured by a circular four-microphone array mounted on the robot, which also estimates the direction of sound arrival, enabling the spatial mapping of acoustic events in the environment. Detected sounds are classified into three categories: environmental (ignored), supervised (requiring human attention), and emergency (requiring immediate action). The system has been implemented as a package in ROS 2, a robotic middleware framework, and validated in real domestic scenarios. Results show that integrating auditory perception allows the robot to adapt its behavior more intelligently and safely to unexpected acoustic events.