CrossMP-SENet: Transformer-Based Cross-Attention for Joint Magnitude-Phase Speech Enhancement
摘要
We propose CrossMP-SENet, a speech enhancement architecture that jointly models magnitude and phase spectra using parallel decoding branches connected via cross-attention. Unlike prior approaches that largely treat magnitude and phase enhancement separately or asymmetrically, our method introduces a dedicated cross-attention block that enables deep, bidirectional interaction between the two domains. This design allows the model to leverage complementary spectral cues more effectively during denoising. We adopt a compressed mask prediction framework for magnitude, paired with a dedicated phase decoder, and design a specialized cross-attention mechanism that facilitates information exchange between these representations. To further improve perceptual quality, we incorporate Perceptual Contrast Stretching (PCS). Our experiments on the VoiceBank + DEMAND corpus show that CrossMP-SENet achieves strong performance with a PESQ score of 3.65 using only 2.64 million parameters, outperforming state-of-the-art models with larger architectures. Additionally, we evaluate Transformer and Mamba-based variants and discover that, despite their recent popularity, Mamba blocks do not consistently surpass Transformer-based designs in this context. All models and code are publicly available at https://github.com/StrangeAlex/CrossMP-SENet , fostering reproducibility and further research.