Cross-View Generalisation in Action Recognition: Feature Design for Transitioning from Exocentric To Egocentric Views

Egocentric action recognition is the ability of identifying human actions from videos taken in a first-person perspective. In the context of human-robot interaction, a robot should understand the actions and behaviors of individuals towards it. However, video data of human interactions taken from a third-person perspective (exocentric view) is significantly more abundant, compared to the egocentric perspective. Thus, we propose an approach to train action recognition models using exocentric datasets, that can be used in run-time to classify egocentric data. Our approach relies on a feature space, based on skeleton pose information, strategically designed to be consistent across exocentric and egocentric domains. The results show that models trained on exocentric data can be used on an egocentric view without any fine-tuning or adaptation. The system proved to generalise well in the unseen first-person domain, achieving an accuracy of 97% on a custom egocentric dataset. This score was obtained after combining several independently trained models into an ensemble, recognising actions in a voting mechanism. Using ensembles allowed for higher accuracy scores, and mitigated the variability between performances of different models, proving to be an advantageous alternative to using a single model.