<p>A feature fusion network integrating a convolutional neural network (CNN) and a Swin Transformer is proposed to enhance representation learning for acoustic scene classification (ASC). Log-mel filter bank energies extracted from audio recordings are first fed into both the CNN branch and the Swin Transformer branch. A feature fusion block is then introduced to serve as a bidirectional bridge between the two branches, enabling effective information exchange. Specifically, the global contextual representations learned by the Swin Transformer branch are conveyed to the CNN branch, while the local features extracted by the CNN branch are progressively transmitted to the patch embedding module to enhance the local representation capability of the Swin Transformer branch. Through this mechanism, the model achieves effective fusion of local and global information and promotes complementary learning between the two branches. Finally, the fused features are aggregated and passed through a feed-forward network (FFN) layer to perform ASC. Experimental results demonstrate that the proposed method achieves accuracies of 85.8% and 79.8% on the DCASE2019 and DCASE2020 datasets, respectively, outperforming existing ASC methods by approximately 4% and 7%.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

A Feature Fusion Network Architecture for Acoustic Scene Classification Using Convolutional Neural Network and Swin Transformer

  • Lin Geng,
  • Dong-Xu Lin,
  • Chun-Dong He

摘要

A feature fusion network integrating a convolutional neural network (CNN) and a Swin Transformer is proposed to enhance representation learning for acoustic scene classification (ASC). Log-mel filter bank energies extracted from audio recordings are first fed into both the CNN branch and the Swin Transformer branch. A feature fusion block is then introduced to serve as a bidirectional bridge between the two branches, enabling effective information exchange. Specifically, the global contextual representations learned by the Swin Transformer branch are conveyed to the CNN branch, while the local features extracted by the CNN branch are progressively transmitted to the patch embedding module to enhance the local representation capability of the Swin Transformer branch. Through this mechanism, the model achieves effective fusion of local and global information and promotes complementary learning between the two branches. Finally, the fused features are aggregated and passed through a feed-forward network (FFN) layer to perform ASC. Experimental results demonstrate that the proposed method achieves accuracies of 85.8% and 79.8% on the DCASE2019 and DCASE2020 datasets, respectively, outperforming existing ASC methods by approximately 4% and 7%.