Despite recent progress in cross-lingual Text-to-Speech (TTS), generating emotionally expressive speech across languages remains a major challenge. Existing cross-lingual TTS systems primarily focus on preserving speaker identity, often neglecting emotional expressiveness. In this paper, we propose CL-EDiff, a cross-lingual emotional TTS framework that integrates high dimensional emotion embeddings extracted from a multilingual speech emotion recognition model. These embeddings are used to condition a diffusion-based prosody predictor that explicitly models pitch, energy, and duration. To enhance emotional fidelity during denoising, we incorporate Emotion-Enhancement Normalization (ETEN) into the diffusion architecture. Additionally, we propose a disentangled Variance Adaptor that enables fine-grained prosody control and alleviates accent-related distortions, thereby improving both emotional expressiveness and naturalness in cross-lingual speech. Experiments on Chinese-English bilingual datasets demonstrate that CL-EDiff produces speech with more accurate emotional expression and natural prosody. Both objective and subjective evaluations show that our method consistently outperforms strong baselines in emotion classification accuracy and perceptual quality. Audio samples are available at: https://cl-ediff.github.io/CL-EDiff/ .

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

CL-EDiff: Cross-Lingual Emotional TTS System Based on Diffusion Model

  • Haiyang Zhou,
  • Zhihua Huang,
  • Bowen Li

摘要

Despite recent progress in cross-lingual Text-to-Speech (TTS), generating emotionally expressive speech across languages remains a major challenge. Existing cross-lingual TTS systems primarily focus on preserving speaker identity, often neglecting emotional expressiveness. In this paper, we propose CL-EDiff, a cross-lingual emotional TTS framework that integrates high dimensional emotion embeddings extracted from a multilingual speech emotion recognition model. These embeddings are used to condition a diffusion-based prosody predictor that explicitly models pitch, energy, and duration. To enhance emotional fidelity during denoising, we incorporate Emotion-Enhancement Normalization (ETEN) into the diffusion architecture. Additionally, we propose a disentangled Variance Adaptor that enables fine-grained prosody control and alleviates accent-related distortions, thereby improving both emotional expressiveness and naturalness in cross-lingual speech. Experiments on Chinese-English bilingual datasets demonstrate that CL-EDiff produces speech with more accurate emotional expression and natural prosody. Both objective and subjective evaluations show that our method consistently outperforms strong baselines in emotion classification accuracy and perceptual quality. Audio samples are available at: https://cl-ediff.github.io/CL-EDiff/ .