Online action detection aims to identify action occurring in the most recent frame of streaming video. Due to its online nature and the presence of abundant background information in streaming videos, online action detection is quite challenging. In this paper, we introduce a solution to the challenging task of online action detection. We propose a gated cross attention module, which is concurrently applied to both the long-term memory encoder and the short-term memory decoder. This module dynamically adjusts attention weights by generating gated tensors and actively participating in attention calculation, thereby facilitating effective information reweighting. Moreover, we integrate background classification with online action detection to bolster action modeling. Besides, an online inference acceleration method is realized for our proposed algorithm. Experimental results demonstrate that our method achieves competitive performance against other state-of-the-art methods.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Online Action Detection with Gated Cross Attention Module

  • Boyu Ying,
  • Wenze Huang,
  • Zhiyong Wang,
  • Weihong Ren,
  • Senming Zhong,
  • Honghai Liu

摘要

Online action detection aims to identify action occurring in the most recent frame of streaming video. Due to its online nature and the presence of abundant background information in streaming videos, online action detection is quite challenging. In this paper, we introduce a solution to the challenging task of online action detection. We propose a gated cross attention module, which is concurrently applied to both the long-term memory encoder and the short-term memory decoder. This module dynamically adjusts attention weights by generating gated tensors and actively participating in attention calculation, thereby facilitating effective information reweighting. Moreover, we integrate background classification with online action detection to bolster action modeling. Besides, an online inference acceleration method is realized for our proposed algorithm. Experimental results demonstrate that our method achieves competitive performance against other state-of-the-art methods.