Learning Causality Under Uncertainty for Egocentric Action Anticipation

Egocentric action anticipation, as the task to infer future actions based on egocentric observations, is very appealing to discover patterns, conducts and, eventually, action causality. In the real world, the observations are usually uncertain, which means they are incomplete and noisy. Consequently, addressing uncertainty implies some kind of optimization. Traditional approaches develop classification tasks to calculate the actions with the highest probability to appear next (maximization problem), but they do not exploit the cause-and-effect relationships. We present here a simple two-step approach. The first step formulates and solves a constraint optimization problem that learns the action causality from a training dataset. Given an observation segment of actions over a new dataset, the second step uses such causality in a matching process to anticipate the highest scored actions. Additionally, this causality can be used to derive the planning action model. The experiments in the EPIC-KITCHENS dataset, a large-scale egocentric benchmark, show that our approach is competitive in terms of accuracy and recall.