<p>Generating realistic 3D dance based on music is a unique and challenging task because real dance is a free and creative form of artistic expression. Human dance styles are diverse and ever-changing. Existing methods fail to perform well on out-of-domain music inputs or to generate movements of new dance styles. Accordingly, in this paper, we propose the Dance Latent Diffusion Model (DanceLDM), an advanced editable dance generation method for creating realistic and diverse 3D dance movements, while providing a text prompt interface for dance movements editing. Inspired by previous studies, we explore multimodal data and multi-task training for dance generation models, and we build a latent diffusion model to compare with existing approaches. Our method allows users to generate dance motions using either music or textual descriptions, while also enabling post-generation editing via text prompts. We evaluate our approach through quantitative and qualitative experiments. In quantitative analysis, DanceLDM achieves superior performance on the Beat Alignment Score (BAS), demonstrating its ability to generate rhythmically aligned dances. In qualitative studies, the participants rated our method as producing more realistic and musically coherent dances on generation and editing compared to existing baselines.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

DanceLDM: latent-based diffusion model for dance generation and editing conditioned on music and text prompt

  • Ming-Cong Su,
  • Wei-Lun Huang,
  • Tse-Yu Pan

摘要

Generating realistic 3D dance based on music is a unique and challenging task because real dance is a free and creative form of artistic expression. Human dance styles are diverse and ever-changing. Existing methods fail to perform well on out-of-domain music inputs or to generate movements of new dance styles. Accordingly, in this paper, we propose the Dance Latent Diffusion Model (DanceLDM), an advanced editable dance generation method for creating realistic and diverse 3D dance movements, while providing a text prompt interface for dance movements editing. Inspired by previous studies, we explore multimodal data and multi-task training for dance generation models, and we build a latent diffusion model to compare with existing approaches. Our method allows users to generate dance motions using either music or textual descriptions, while also enabling post-generation editing via text prompts. We evaluate our approach through quantitative and qualitative experiments. In quantitative analysis, DanceLDM achieves superior performance on the Beat Alignment Score (BAS), demonstrating its ability to generate rhythmically aligned dances. In qualitative studies, the participants rated our method as producing more realistic and musically coherent dances on generation and editing compared to existing baselines.