Hybrid Methods for Real-Time Video Sequence Identification of Human Soccer Referee Signals

We evaluate several real-time machine learning and hybrid symbolic algorithms to identify body-gestures of Human Soccer Referees as part of the 2022 RoboCup Soccer Standard Platform League (SPL) Visual Referee Challenge. This challenge, in part, encourages development of software for effective human-robot interactions as the Nao must interpret and respond in real-time to human referee’s gestures, so RoboCup can progress towards it’s 2050 goal. We train and evaluate the real-time accuracy and speed of two Deep Convolutional Neural Networks architectures, OpenPose and Stacked HourGlass, against our novel hybrid machine-learning and symbolic approach on the resource constrained SoftBank Nao V6. Our results show that there remains an important trade-off between the single-image accuracy of Deep Networks, and the execution speed of hybrid when evaluating across a video sequence. Our hybrid method achieved a strong result in 2022 SPL “Visual Referee Challenge”. Our work has been motivated by the requirements for methods in human-robot interaction, where the human perspective of a robot’s behaviour strongly influences the evaluation criteria of software.