A Benchmark of Egocentric Scene Graph Prediction Methods for Understanding Human-Object Interactions
摘要
Egocentric videos captured by wearable cameras provide a rich, first-person view of human-object interactions, offering unique insights into how people perform everyday activities. However, understanding long-form egocentric video remains challenging due to the complex temporal structure and dynamic nature of this data. In this work, we address the task of Egocentric Action Scene Graph (EASG) generation, which constructs structured graphs that represent hands, objects, and their semantic relationships over time. Unlike previous methods that rely on ground-truth object annotations, we evaluate models in two settings: in the presence of ground-truth object annotations, and in the absence of such annotations, where algorithms rely on predicted objects. We evaluate multiple methods, including transformer-based and fully connected models, for predicting temporally coherent and semantically rich scene graphs. The benchmarked approaches capture the hierarchical and compositional aspects of first-person behavior, potentially enabling improved interpretation and generalization for downstream tasks such as action anticipation and video summarization.