Generating dance animations from music presents significant challenges in artificial intelligence, requiring systems to capture the complex mapping between audio features and human motion. We introduce a novel multi-stage decomposition-recombination network for music-driven dance synthesis that addresses two critical limitations in existing approaches. First, unlike end-to-end deep learning models that directly transform music into posture data—often resulting in unrealistic movements—our approach decomposes both music and dance into fundamental units and learns mappings between them, preserving the naturalness of human motion. Second, we establish that style-specific training delivers more distinctive and stylistically coherent choreography than mixed-style approaches. Our framework implements a three-stage process: (1) an accumulation stage that constructs music and dance unit dictionaries through clustering techniques, (2) a learning stage that trains style-specific mapping models between these units, and (3) a creation stage that recombines units to generate coherent dance sequences. Quantitative evaluations show that our method outperforms the baseline approaches by 26.9% in Fréchet Inception Distance (FID) and 13.1% in music-dance correspondence scores. Qualitative analysis confirms our framework’s ability to generate dance sequences with both improved temporal coherence and clear stylistic characteristics across breaking, hip-hop, locking, and street jazz styles. The proposed approach offers a significant advancement in realistic, style-specific music-to-dance synthesis.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Style-Aware Music-to-Dance Generation via Multi-Stage Unit Decomposition and Recombination

  • Yufei Gao,
  • Qian Wu,
  • Keren He,
  • Jinjia Zhou

摘要

Generating dance animations from music presents significant challenges in artificial intelligence, requiring systems to capture the complex mapping between audio features and human motion. We introduce a novel multi-stage decomposition-recombination network for music-driven dance synthesis that addresses two critical limitations in existing approaches. First, unlike end-to-end deep learning models that directly transform music into posture data—often resulting in unrealistic movements—our approach decomposes both music and dance into fundamental units and learns mappings between them, preserving the naturalness of human motion. Second, we establish that style-specific training delivers more distinctive and stylistically coherent choreography than mixed-style approaches. Our framework implements a three-stage process: (1) an accumulation stage that constructs music and dance unit dictionaries through clustering techniques, (2) a learning stage that trains style-specific mapping models between these units, and (3) a creation stage that recombines units to generate coherent dance sequences. Quantitative evaluations show that our method outperforms the baseline approaches by 26.9% in Fréchet Inception Distance (FID) and 13.1% in music-dance correspondence scores. Qualitative analysis confirms our framework’s ability to generate dance sequences with both improved temporal coherence and clear stylistic characteristics across breaking, hip-hop, locking, and street jazz styles. The proposed approach offers a significant advancement in realistic, style-specific music-to-dance synthesis.