Personalized and Age-Appropriate Speech Synthesis for Nonverbal Children Using Diffusion-Based Voice Cloning and Parametric Vocoding
摘要
Children with severe neurodevelopmental conditions, including autism spectrum disorder, frequently struggle with verbal expression. This inability creates significant communication challenges and can impact their personal identity development. While Augmentative and Alternative Communication (AAC) technologies offer speech capabilities, they typically employ impersonal voice synthesis. This paper presents a system that generates a personalized and age-appropriate voice for nonverbal children by fitting a speech synthesis model. Our approach uses a Zero-Shot Diffusion Transformer to clone vocal timbre from a short audio sample of a close relative. A parametric vocoder then precisely modifies acoustic parameters, such as the fundamental frequency, to match the target child’s age and synthesizes a corpus of functional sentences on an ESP32-based portable device. In subjective listening tests with 20 native speakers, the system achieved high natural speech with a mean opinion score of 4.41 and high similarity to the target speaker of 4.4. This work demonstrates a computationally accessible method as a tool that can restore vocal identity and autonomy, thereby improving generic speech-to-speech systems. By providing a unique voice, this technology has the potential to significantly improve social integration and psychological well-being, representing a crucial step towards more human-centered clinical AAC solutions.