Real-time speech enhancement is crucial for applications such as VoIP calls and hearing aids, which require low-latency noise suppression on resource-constrained devices. We present a causal speech enhancement model based on Facebook’s Denoiser (a Demucs-inspired architecture), modified for deployment as an ONNX model. Key contributions include fixing the input to 2 s of audio at 16 kHz (32,000 samples), removing dynamic input length computation by padding to a constant length (32,085 samples), and trimming outputs accordingly during the training process. The model uses a convolutional encoder-decoder architecture with a gated LSTM bottleneck, trained on the CSTR VCTK Corpus+DEMAND (Valentini) dataset with data augmentation, and fine-tuned on the Microsoft Scalable Noisy Speech Dataset (MS-SNSD). The enhanced model achieves high speech quality, achieving results that match the state-of-the-art benchmarks for causal speech enhancement. The proposed model achieves PESQ 2.996 (vs. 3.07 for the original Denoiser), STOI 0.944 (vs. 0.95), CSIG 4.016, CBAK 3.453, and COVL 3.525 on the Valentini test set on par with state-of-the-art causal methods while remaining compatible with ONNX Opset 9. We also implemented a model based on a GRU bottleneck and observed a slight performance loss for offline processing, but better results during the online evaluation compared to the LSTM based model.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Compact Real-Time Speech Enhancement for ONNX

  • Andrei Filip,
  • Honorius Gâlmeanu,
  • Alexandru Drîmbărean

摘要

Real-time speech enhancement is crucial for applications such as VoIP calls and hearing aids, which require low-latency noise suppression on resource-constrained devices. We present a causal speech enhancement model based on Facebook’s Denoiser (a Demucs-inspired architecture), modified for deployment as an ONNX model. Key contributions include fixing the input to 2 s of audio at 16 kHz (32,000 samples), removing dynamic input length computation by padding to a constant length (32,085 samples), and trimming outputs accordingly during the training process. The model uses a convolutional encoder-decoder architecture with a gated LSTM bottleneck, trained on the CSTR VCTK Corpus+DEMAND (Valentini) dataset with data augmentation, and fine-tuned on the Microsoft Scalable Noisy Speech Dataset (MS-SNSD). The enhanced model achieves high speech quality, achieving results that match the state-of-the-art benchmarks for causal speech enhancement. The proposed model achieves PESQ 2.996 (vs. 3.07 for the original Denoiser), STOI 0.944 (vs. 0.95), CSIG 4.016, CBAK 3.453, and COVL 3.525 on the Valentini test set on par with state-of-the-art causal methods while remaining compatible with ONNX Opset 9. We also implemented a model based on a GRU bottleneck and observed a slight performance loss for offline processing, but better results during the online evaluation compared to the LSTM based model.