<p>Although the field of deep learning has improved video-based dance classification, traditional models are computationally inefficient because they process redundant frames, and do not have the capability to center on discriminative key moments. In order to fill this gap, the present paper proposes the Reinforcement-based Attentive Temporal Sampling (RATS) framework. RATS proposes a new algorithmic framework in that it develops the classification as a sequential decision-making process. It is a three-part modular structure, which includes a custom feature extraction pipeline (based on 3D Convolutional Neural Networks (3DCNN)) to learn rich visual-motion representations, a Deep Q-Network (DQN) agent with Bidirectional Long Short-Term Memory (BiLSTM) memory to learn the optimal movement policy in the video, and a final classification head that predicts the dance style based on the state summary presented by the BiLSTM. This approach, while significantly reducing the computational complexity and focusing on important frames, significantly improves the accuracy of the model in recognizing complex dance styles; in such a way that in the evaluation on the Let’s Dance dataset, it achieved the accuracy of 92.1%, showing a 3.9% increase in accuracy and a 4% improvement in the F-measure, which indicates the outstanding efficiency and effectiveness of RATS.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A deep reinforcement learning approach to dance movement analysis

  • Peili Yin,
  • Xin Li

摘要

Although the field of deep learning has improved video-based dance classification, traditional models are computationally inefficient because they process redundant frames, and do not have the capability to center on discriminative key moments. In order to fill this gap, the present paper proposes the Reinforcement-based Attentive Temporal Sampling (RATS) framework. RATS proposes a new algorithmic framework in that it develops the classification as a sequential decision-making process. It is a three-part modular structure, which includes a custom feature extraction pipeline (based on 3D Convolutional Neural Networks (3DCNN)) to learn rich visual-motion representations, a Deep Q-Network (DQN) agent with Bidirectional Long Short-Term Memory (BiLSTM) memory to learn the optimal movement policy in the video, and a final classification head that predicts the dance style based on the state summary presented by the BiLSTM. This approach, while significantly reducing the computational complexity and focusing on important frames, significantly improves the accuracy of the model in recognizing complex dance styles; in such a way that in the evaluation on the Let’s Dance dataset, it achieved the accuracy of 92.1%, showing a 3.9% increase in accuracy and a 4% improvement in the F-measure, which indicates the outstanding efficiency and effectiveness of RATS.