The main goal of the current study was to test the TTS model Tacotron2 for generating intonation models according to N.B.Volskaya’s classification. In order to achieve this goal, we consecutively solved various tasks. First, we selected the fully annotated corpus of Russian monological speech (CORPRESS) for the analysis, due to the intonation model markups and a thourough segmentation, as well as to the quality of recorded speech (the corpus consists of professional actors’ and speakers’ recordings). From this speech corpus we choose 4 male speakers recordings. Then, we modified the architecture of the Tacotron2 model in a way to face the challenge of intonation model classification and made data preprocessing, that included data preliminary statistical analysis and preliminary training, which showed the need of data augmentation for creating a well-equilibrate material in training and validating datasets. After this, we produced an additional training, which showed good results. Two auditory perceptual experiments were conducted. First experiment consisted of MOS evaluation test and resulted at 4.027 points. Second experiment provided data on sentence type recognition. A consecutive comparative acoustic and expert auditory analysis of natural and generated pitch patterns showed that various intonation models can be successfully reproduced, although the most resemblance is noticed for the models with an even tone. The results obtained provide new information on intonation synthesis perspectives and demonstrate a huge potential of using N.B. Volskaya’s system for the annotation of the training dataset in order to obtain an effective synthesis of functional intonation models in Russian.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Effectiveness of Tacotron2 for Intonation Model Synthesis in Russian

  • Anastasiia Sherban,
  • Uliana Kochetkova

摘要

The main goal of the current study was to test the TTS model Tacotron2 for generating intonation models according to N.B.Volskaya’s classification. In order to achieve this goal, we consecutively solved various tasks. First, we selected the fully annotated corpus of Russian monological speech (CORPRESS) for the analysis, due to the intonation model markups and a thourough segmentation, as well as to the quality of recorded speech (the corpus consists of professional actors’ and speakers’ recordings). From this speech corpus we choose 4 male speakers recordings. Then, we modified the architecture of the Tacotron2 model in a way to face the challenge of intonation model classification and made data preprocessing, that included data preliminary statistical analysis and preliminary training, which showed the need of data augmentation for creating a well-equilibrate material in training and validating datasets. After this, we produced an additional training, which showed good results. Two auditory perceptual experiments were conducted. First experiment consisted of MOS evaluation test and resulted at 4.027 points. Second experiment provided data on sentence type recognition. A consecutive comparative acoustic and expert auditory analysis of natural and generated pitch patterns showed that various intonation models can be successfully reproduced, although the most resemblance is noticed for the models with an even tone. The results obtained provide new information on intonation synthesis perspectives and demonstrate a huge potential of using N.B. Volskaya’s system for the annotation of the training dataset in order to obtain an effective synthesis of functional intonation models in Russian.