Speaker-Aware Speech Enhancement with Joint Optimization for Low SNR Environments
摘要
This paper proposes a speaker-aware speech enhancement joint training framework for low SNR environments. Traditional cascaded approaches suffer from error accumulation and conflicting objectives between speech enhancement and speaker recognition tasks. Our framework introduces three key innovations: a Feature Bridging Encoder connecting both tasks, an adversarial discriminator ensuring the preservation of speaker characteristics, and a progressive three-stage training strategy enabling balanced optimization. Experiments on the VoxCeleb1 dataset with various noise conditions demonstrate that our approach consistently outperforms traditional cascaded and multi-task learning approaches across all SNR levels. The performance advantage is particularly significant in challenging low SNR environments, where speaker feature preservation is most difficult. Ablation studies confirm the essential role of each component, with the Feature Bridging Encoder and adversarial learning mechanism proving critical to the framework’s success. This work provides a promising solution for speech processing systems operating in adverse acoustic environments where both speech quality and speaker identity are important.