The majority of Chinese music is disseminated through Jianpu, a distinctive musical notation system that uses numbers to denote pitches and underlines to indicate note durations. Despite its widespread use, Jianpu lacks robust optical recognition procedures, primarily due to the absence of large, annotated datasets. To address this challenge, we developed a comprehensive framework for the digital representation and rendering of Jianpu notation, aimed at constructing a synthetic dataset for Optical Music Recognition (OMR). Our framework employs a JSON-based representation system for Jianpu scores, enabling seamless conversion to and from other digital formats like MIDI and MusicXML. We implemented a hierarchical, highly parameterized algorithm for score layout and image rendering, ensuring diversity in the generated sheet images and the ability to produce corresponding semantic annotations, such as bounding boxes. By leveraging this framework, we created a synthetic dataset comprising images of Jianpu music sheets, both from existing and randomly generated scores. This dataset was subsequently used to train a convolutional neural network for OMR tasks. The trained network demonstrated outstanding performance when applied to real-world scanned Jianpu sheets, significantly reducing the need for manual annotation.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Framework for the Digital Representation and Rendering of Chinese Jianpu Notation for Constructing a Synthetic OMR Dataset

  • Fan Bu,
  • Rongfeng Li,
  • Zijin Li,
  • Ya Li,
  • Linfeng Fan

摘要

The majority of Chinese music is disseminated through Jianpu, a distinctive musical notation system that uses numbers to denote pitches and underlines to indicate note durations. Despite its widespread use, Jianpu lacks robust optical recognition procedures, primarily due to the absence of large, annotated datasets. To address this challenge, we developed a comprehensive framework for the digital representation and rendering of Jianpu notation, aimed at constructing a synthetic dataset for Optical Music Recognition (OMR). Our framework employs a JSON-based representation system for Jianpu scores, enabling seamless conversion to and from other digital formats like MIDI and MusicXML. We implemented a hierarchical, highly parameterized algorithm for score layout and image rendering, ensuring diversity in the generated sheet images and the ability to produce corresponding semantic annotations, such as bounding boxes. By leveraging this framework, we created a synthetic dataset comprising images of Jianpu music sheets, both from existing and randomly generated scores. This dataset was subsequently used to train a convolutional neural network for OMR tasks. The trained network demonstrated outstanding performance when applied to real-world scanned Jianpu sheets, significantly reducing the need for manual annotation.