<p>Predominant instrument recognition in polyphonic music is challenging due to overlapping spectral characteristics of multiple instruments. Most existing approaches treat spectrograms as images and apply convolutional or transformer-based architectures, often ignoring the structural relationships among time–frequency regions. To address this limitation, this paper proposes a framework that learns graph representations from Mel-spectrogram segments for predominant instrument recognition. The process involves constructing a graph from the Mel-spectrogram of an audio file using a trained deep convolutional neural network (CNN). The resulting graph is then processed by graph learning frameworks such as graph convolution neural networks (GCN), ChebNet (graph convolutions using Chebyshev polynomials), GraphSAGE (Graph-SAmple and aggreGatE), and graph attention networks (GAT) for classification. The proposed method is assessed using the Instrument Recognition Music Audio Signal (IRMAS) dataset, in which the training phase comprises fixed-length segments featuring just one predominant instrument, whereas the model is tested for its ability to identify an arbitrary number of multiple predominant instruments of variable-length. To improve model generalization and resilience, a range of data augmentation techniques is utilized, including polyphonic mixing, SpecAugment, and music audio synthesis via Wave Generative Adversarial Networks (WaveGAN). Among the evaluated graph models, Spec2GAT provides the best overall performance with relatively low additional computational complexity. This work not only advances the field of predominant instrument recognition but also paves the way for future research in other similar classification tasks, where graph-based methods can be utilized for complex audio analysis.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Learning graph representations from Mel-spectrogram segments for predominant instrument recognition in polyphonic music

  • Riya Rajeev,
  • Lekshmi Chandrika Reghunath,
  • Noumida A.,
  • Rajeev Rajan

摘要

Predominant instrument recognition in polyphonic music is challenging due to overlapping spectral characteristics of multiple instruments. Most existing approaches treat spectrograms as images and apply convolutional or transformer-based architectures, often ignoring the structural relationships among time–frequency regions. To address this limitation, this paper proposes a framework that learns graph representations from Mel-spectrogram segments for predominant instrument recognition. The process involves constructing a graph from the Mel-spectrogram of an audio file using a trained deep convolutional neural network (CNN). The resulting graph is then processed by graph learning frameworks such as graph convolution neural networks (GCN), ChebNet (graph convolutions using Chebyshev polynomials), GraphSAGE (Graph-SAmple and aggreGatE), and graph attention networks (GAT) for classification. The proposed method is assessed using the Instrument Recognition Music Audio Signal (IRMAS) dataset, in which the training phase comprises fixed-length segments featuring just one predominant instrument, whereas the model is tested for its ability to identify an arbitrary number of multiple predominant instruments of variable-length. To improve model generalization and resilience, a range of data augmentation techniques is utilized, including polyphonic mixing, SpecAugment, and music audio synthesis via Wave Generative Adversarial Networks (WaveGAN). Among the evaluated graph models, Spec2GAT provides the best overall performance with relatively low additional computational complexity. This work not only advances the field of predominant instrument recognition but also paves the way for future research in other similar classification tasks, where graph-based methods can be utilized for complex audio analysis.