Improving Speech Perceptual Quality and Intelligibility Through Sub-band Temporal Envelope Characteristics

In the speech enhancement (SE) model, using auxiliary loss based on acoustic parameters can improve enhancement effects. However, currently used acoustic parameters focus on frequency domain information, neglecting the importance of time domain information. This study explores the effects of training the SE model utilizing temporal envelope parameters as auxiliary losses on perceptual quality and intelligibility, primarily focusing on accurately extracting temporal envelope parameters and optimizing losses. Initially, the effectiveness and robustness of the temporal envelope parameters in optimizing the SE model were verified. Subsequently, a temporal envelope loss based on sub-band weighted (ENVLoss) is proposed based on the perceptual characteristics of human hearing. Then, a method for constructing a joint loss function is proposed, integrating temporal and spectral acoustic features to promote the enhancement effect. Experimental results show that temporal envelope characteristics improve both the time domain and the time-frequency domain SE models. Compared to other acoustic parameter losses, the SE model using a sub-band temporal envelope auxiliary loss shows improvement in the PESQ, STOI, and MOS estimation metrics.