C2-MAGIC: Chord-Controllable Multi-track Accompaniment Generation with Interpretability and Creativity

Musical notes are usually grouped into chords to generate aesthetically pleasing music, which is a special characteristic in symbolic music modeling. Good manipulation of chords not only brings harmony to music but also makes interactive music generation more manageable. However, prior attempts at chord-conditioned music generation have several limits: 1) Most previous work only considers chords as auxiliary tokens by adding them explicitly to the original conditional sequence, nevertheless, this simple design doesn’t provide enough information and isn’t capable of gaining good control over chords. 2) Some research reports good control of chords in terms of analogy but is limited under the context of single-track music of limited length. 3) Some achieve good matches of chords, yet at the cost of other evaluation metrics. To overcome these limits, we propose a novel architecture under the framework of multi-task learning, which additionally learns a latent chord representation used as extra contextual information for the multi-track accompaniment generation task. We evaluate our model both on the public LMD dataset and a private dataset of pop music. Experiments show that our model largely outperforms the state-of-the-art multi-track accompaniment generation model in terms of chord control ability and further improves the validation perplexity of the accompaniment generation task. Extensive objective and subjective studies also demonstrate the effectiveness of our approach. Furthermore, we show that our model brings extra interpretability and creativity through various inference experiments.