Scholarlink

<p>While there has been substantial progress in temporal action segmentation, the challenge to generalize to unseen views remains unaddressed. Hence, we define a protocol for unseen view action segmentation where camera views for evaluating the model are unavailable during training. This includes changing from top-frontal views to a side view or even more challenging from exocentric to egocentric views. Furthermore, we present an approach for temporal action segmentation that tackles this challenge. Our approach leverages a shared representation at both the sequence and segment levels to reduce the impact of view differences during training. We achieve this by introducing a sequence loss and an action loss, which together facilitate consistent video and action representations across different views. The evaluation on the Assembly101, IkeaASM, and EgoExoLearn datasets demonstrate significant improvements, with a <InlineEquation ID="IEq1"> <EquationSource Format="TEX">\(12.8\%\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>12.8</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> increase in F1@50 for unseen exocentric views and a substantial <InlineEquation ID="IEq2"> <EquationSource Format="TEX">\(54\%\)</EquationSource> <EquationSource Format="MATHML"><math> <mrow> <mn>54</mn> <mo>%</mo> </mrow> </math></EquationSource> </InlineEquation> improvement for unseen egocentric views.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Towards Generalizing Temporal Action Segmentation to Unseen Views

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年3月9日

查看全文
  • Emad Bahrami,
  • Olga Zatsarynna,
  • Gianpiero Francesca,
  • Juergen Gall

摘要

While there has been substantial progress in temporal action segmentation, the challenge to generalize to unseen views remains unaddressed. Hence, we define a protocol for unseen view action segmentation where camera views for evaluating the model are unavailable during training. This includes changing from top-frontal views to a side view or even more challenging from exocentric to egocentric views. Furthermore, we present an approach for temporal action segmentation that tackles this challenge. Our approach leverages a shared representation at both the sequence and segment levels to reduce the impact of view differences during training. We achieve this by introducing a sequence loss and an action loss, which together facilitate consistent video and action representations across different views. The evaluation on the Assembly101, IkeaASM, and EgoExoLearn datasets demonstrate significant improvements, with a \(12.8\%\) 12.8 % increase in F1@50 for unseen exocentric views and a substantial \(54\%\) 54 % improvement for unseen egocentric views.