Western music theory concepts underlie the compositional processes of much of the music consumed today and consequently serve as the conceptual underpinnings of much of the training data for generative music models. Applying the interpretability technique of probing, Wei et al. [58] studied how well basic musical concepts such as scales, time signatures, and simple triadic harmonic progressions are encoded in the latent spaces of the Jukebox and MusicGen generative models. Through constructing datasets isolating these concepts, they trained simple classifiers to classify elements of these concepts based on their latent representations. We extend their work by creating datasets focused on five higher-level and more advanced music theory concepts: polyrhythms, dynamics, seventh chords, mode mixture, and secondary dominants. Performing experiments with the same Jukebox and MusicGen models, we train classifiers on the datasets’ latent representations. We find that these advanced concepts are encoded in their latent spaces and that secondary dominant classification is the most difficult of our probing tasks.