Social Perception and Scene Awareness in Human-Robot Interaction

This paper introduces various aspects of social perception skills and scene awareness for interactive robots. The low-level audio-visual perceptual cues e.g., interruption events, eye contact, speech energy, etc. have been fused to make high-level interpretations e.g., a person’s willingness to maintain human-robot interaction in social settings. The detection of objects in the scene helps generate context-aware queries or responses for the robot. The overall behavior of the interactive robot, Ameca, has been generated in the form of speech and body language. In this work, a fusion of rule-based and caption-based speech generation has been incorporated. The experimental results in human-robot interaction scenarios show promising outcomes as far as the robot’s ability to perceive and react to social and environmental cues.