Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) are two prominent generative models that excel in different aspects of generative tasks of creating new content, such as images, text, or music using data and patterns. VAEs are known for their capability to learn smooth and probabilistic latent representations, which enable structured generation, but they often produce blurry and unrealistic outputs due to their inherent loss function. On the other hand, GANs generate sharp and visually appealing results by utilizing a discriminator to refine output but suffer from mode collapse, limiting their diversity. This paper proposes a novel hybrid architecture, the Variational Cyclic Generative Adversarial Network (VarCGAN), to transfer the musical style of a song from one genre to another. VarCGAN combines the latent space modeling capabilities of VAEs with the adversarial optimization of GANs to overcome the limitations of each approach. The model introduces a cyclic adversarial loss, which ensures consistency and realism in style-transferred outputs while preserving the original song’s musical essence. Furthermore, the hybrid design enables the generation of diverse variations of genre-transferred songs, capturing subtle stylistic features of the target genre. The proposed approach is evaluated on the GTZAN dataset, focusing on style transfers between classical, jazz, hip-hop, and rock genres. To the best of our knowledge, VarCGAN is the first framework to utilize this hybrid methodology for music genre style transfer, presenting a significant advancement in music composition and genre transformations.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

VarCGAN: Variational Cyclic Generative Adversarial Network For Music Genre Style Transfer

  • Pooja Singh,
  • Dhruv Mishra,
  • Ankita Khandelwal

摘要

Variational Autoencoders (VAEs) and Generative Adversarial Networks (GANs) are two prominent generative models that excel in different aspects of generative tasks of creating new content, such as images, text, or music using data and patterns. VAEs are known for their capability to learn smooth and probabilistic latent representations, which enable structured generation, but they often produce blurry and unrealistic outputs due to their inherent loss function. On the other hand, GANs generate sharp and visually appealing results by utilizing a discriminator to refine output but suffer from mode collapse, limiting their diversity. This paper proposes a novel hybrid architecture, the Variational Cyclic Generative Adversarial Network (VarCGAN), to transfer the musical style of a song from one genre to another. VarCGAN combines the latent space modeling capabilities of VAEs with the adversarial optimization of GANs to overcome the limitations of each approach. The model introduces a cyclic adversarial loss, which ensures consistency and realism in style-transferred outputs while preserving the original song’s musical essence. Furthermore, the hybrid design enables the generation of diverse variations of genre-transferred songs, capturing subtle stylistic features of the target genre. The proposed approach is evaluated on the GTZAN dataset, focusing on style transfers between classical, jazz, hip-hop, and rock genres. To the best of our knowledge, VarCGAN is the first framework to utilize this hybrid methodology for music genre style transfer, presenting a significant advancement in music composition and genre transformations.