Real Time Voice Cloning Using Enhanced TTS and Vocoder
摘要
Voice Synthesis is an Complex field on which Researchers always work. Over the time different advances are been observed in this field for reducing time, complexity, resources required. This research presents a comprehensive voice cloning system that combines state-of-the-art deep learning architectures to achieve high-fidelity speech synthesis with minimal training data. Our approach integrates Tacotron2 for mel-spectrogram prediction, HiFi-GAN as a neural vocoder, and Whisper for improved speech recognition and transcription preprocessing. The system demonstrates remarkable capability in capturing speaker-specific voice characteristics, prosody, and emotional inflections while maintaining natural-sounding speech output. Furthermore, the inclusion of Whisper for preprocessing has improved the system’s robustness to noisy input audio and enhanced phonetic alignment accuracy, resulting in clearer articulation and more consistent prosody in synthesized speech. Our findings suggest that this integrated architecture represents a promising direction for personalized speech synthesis applications across domains including assistive technologies, content creation, and human-computer interaction systems.