A Progressive Learning Approach for Sound Event Detection with Temporal and Spectral Features Fusion

Sound Event Detection (SED) has wide applications for real world systems, including automatic surveillance, smart home devices, and intelligent automobiles. While recent works in SED have achieved significant performance improvements through fine-tuning pre-trained frame-wise audio tagging (AT) models, bridging the gap between AT and SED tasks, a common limitation is their exclusive reliance on spectral features for input. This leads to a challenge for precise sound event localization. To address this issue, we proposed a novel Temporal Mask Model (TMM) extracting temporal features, integrated with the Bidirectional Encoder representation from Audio Transformers and CNN (BEATs-CNN) framework which extracts spectral features. These two types of features are fused with a progressive learning strategy, and consequently fed into a Bidirectional Gated Recurrent Unit (Bi-GRU) to generate predictions. Through extensive experimentation, we demonstrate that our approach surpasses the reported State-Of-The-Art (SOTA) model in Polyphonic Sound Detection Score-scenario1 (PSDS1) and achieves a comparable result in Polyphonic Sound Detection Score-scenario2 (PSDS2) on the DCASE Challenge Task 4.