Action Recognition System Integrating Motion and Object Detection

In this paper, we present a novel action recognition system based on the integration of information from two separate modules. The first module is responsible for motion detection and categorization. The second module is an instance segmentation module that recognizes objects and their positions in the scene. The information from both modules is integrated in the third module that recognizes the actions based on motion and object detection. Compared to the traditional systems based on motion detection, we are able to recognize fake actions (gestures) where no contextual objects are presented in the scene. Moreover, we detect the average motion speed of contextual objects to increase the precision of detected actions. We create a dataset of eight action types that include assembly actions with tools and also corresponding fake actions that have similar motion but where no tools are used. Our recognition system achieves 95.21% accuracy in this dataset compared to 85.52% for a system based on motion detection only. We demonstrate that combining data from two different sources can improve the overall results of the action recognition task. Our recognition system can be adopted in real world tasks to distinguish between real actions and gestures.