<p>Speech enhancement is a fundamental task in many speech processing systems. In recent years, researchers have increasingly focused on boosting performance by capturing long-range contextual dependencies within speech signals. A widely adopted solution is multi-stage learning, where several deep learning components are arranged in sequence to refine the results step by step. Likewise, attention-based mechanisms have proven highly effective in improving speech quality, especially when combined with convolutional neural networks (CNNs). Nevertheless, most conventional attention designs rely on fully connected and convolutional operations, which significantly increase both the number of parameters and computational demands. To address this, the present study introduces a multi-stage speech enhancement framework that integrates Squeeze Temporal Convolutional Modules (STCM) with exponentially increasing dilation rates and a Neural-Free Attention (NFA) mechanism at each stage. At every phase, an intermediate estimate is generated and further refined in subsequent phases, with a Feature Fusion Module (FFM) reintroducing original information at the start of each stage. This design allows the intermediate outputs to undergo step-by-step improvements through successive STCMs, ultimately enabling precise spectral estimation. The NFA, a lightweight and easily integrable component, enhances the model’s ability to capture fine-grained energy distributions across frequency channels by generating attention weights with a trainable Gaussian function. The proposed system is evaluated on the VCTK and LibriSpeech datasets, showing superior performance compared to state-of-the-art deep learning methods in terms of PESQ, STOI, CSIG, CBAK, and COVL metrics.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speech enhancement using neural free attention with multi-stage squeeze temporal convolutional networks

  • Chaitanya Jannu,
  • Manaswini Burra,
  • Sunny Dayal Vanambathina,
  • Veeraswamy Parisae

摘要

Speech enhancement is a fundamental task in many speech processing systems. In recent years, researchers have increasingly focused on boosting performance by capturing long-range contextual dependencies within speech signals. A widely adopted solution is multi-stage learning, where several deep learning components are arranged in sequence to refine the results step by step. Likewise, attention-based mechanisms have proven highly effective in improving speech quality, especially when combined with convolutional neural networks (CNNs). Nevertheless, most conventional attention designs rely on fully connected and convolutional operations, which significantly increase both the number of parameters and computational demands. To address this, the present study introduces a multi-stage speech enhancement framework that integrates Squeeze Temporal Convolutional Modules (STCM) with exponentially increasing dilation rates and a Neural-Free Attention (NFA) mechanism at each stage. At every phase, an intermediate estimate is generated and further refined in subsequent phases, with a Feature Fusion Module (FFM) reintroducing original information at the start of each stage. This design allows the intermediate outputs to undergo step-by-step improvements through successive STCMs, ultimately enabling precise spectral estimation. The NFA, a lightweight and easily integrable component, enhances the model’s ability to capture fine-grained energy distributions across frequency channels by generating attention weights with a trainable Gaussian function. The proposed system is evaluated on the VCTK and LibriSpeech datasets, showing superior performance compared to state-of-the-art deep learning methods in terms of PESQ, STOI, CSIG, CBAK, and COVL metrics.