Real Time State Detection of a Foosball Game Using CNN-Based Computer Vision

Abstractions of the game of football serve as well-known challenges in AI research. A particularly accessible abstraction is the game of Foosball where one team is operated by an AI agent while the other side is controlled by humans. In Foosball, the dynamics can be described by a few descriptive parameters, namely the shift and rotation of the corresponding rods plus the position of the ball. In this work, we present a Computer Vision based real time game state detector in a real-world setup with an automated Foosball table (constructed by Bosch Rexroth AG). More precisely, we train an object detector network based on YOLOX to detect the positions of the figures and an image regressor network based on ResNet18 to predict the rotation angles. For the derivation of the training data we propose a semi-supervised labeling approach based on classical Computer Vision. We evaluate the proposed approach and find that our methodology works in the sense of a proof of concept. The resulting prototype generated promising results with low inference times meeting our real time requirement of 60 fps.