Automatic singing transcription (AST), which refers to the process of inferring the onset, offset, and pitch from the singing audio, is of great significance in music information retrieval. In recent years, methods based on deep neural networks have made significant progress in the field of AST. CNNs, RNNs, transformers, and even object detection methods have all demonstrated excellent performance in the AST domain. It is found that spectrum note detection has obvious two-dimensional long sequence features in AST scenarios. Therefore, Mamba is introduced into AST tasks for the first time, and the Mamba-based AST model, MusicMamba, is proposed. We specifically modify the 2D Selective Scan (SS2D) structure to achieve better model performance. We conducted comparative tests with mainstream AST models on the SSVD3.0, ISMIR2014, and MIR-ST500 datasets, and MusicMamba demonstrated outstanding performance across all datasets.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MusicMamba: A Mamba-Based Automatic Singing Transcription Model

  • Mengqiao Chen,
  • Tianrui Li,
  • Qikai He,
  • Qian Huang,
  • Wei Xu

摘要

Automatic singing transcription (AST), which refers to the process of inferring the onset, offset, and pitch from the singing audio, is of great significance in music information retrieval. In recent years, methods based on deep neural networks have made significant progress in the field of AST. CNNs, RNNs, transformers, and even object detection methods have all demonstrated excellent performance in the AST domain. It is found that spectrum note detection has obvious two-dimensional long sequence features in AST scenarios. Therefore, Mamba is introduced into AST tasks for the first time, and the Mamba-based AST model, MusicMamba, is proposed. We specifically modify the 2D Selective Scan (SS2D) structure to achieve better model performance. We conducted comparative tests with mainstream AST models on the SSVD3.0, ISMIR2014, and MIR-ST500 datasets, and MusicMamba demonstrated outstanding performance across all datasets.