Multi-label audio classification is a challenging task due to complex soundscapes, overlapping events, and class imbalance, particularly in fine-grained biodiversity monitoring scenarios. In this study, we present CNNDualMamba, a new neural architecture made for multi-label anuran species classification. Our model combines three main parts: First, it uses a convolutional neural network as the backbone to efficiently extract time-frequency features. Second, it introduces a loss function that’s more suited to multi-label tasks. This function combines zero-bounded log-sum-exp and pairwise rank-based loss. Third, it designs a new dual-branch Mamba module. This module models time and frequency dependencies separately. In addition, we carry out label-aware data augmentation to solve the data imbalance problem. Experimental results on AnuraSet show that our model outperforms existing baselines and achieves an overall F1 score of 60.1%. For three subsets, our best model achieves an F1 score of 86.5%, 81.1%, and 49.8% for frequent, common, and rare classes, respectively.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Dual-Branch Mamba Based Multi-label Anuran Species Classification

  • Yuji Wang,
  • Longhui Zhao,
  • Juan Gabriel Colonna,
  • Jia Tang,
  • Zujie Kang,
  • Faming Zhang,
  • Jie Xie

摘要

Multi-label audio classification is a challenging task due to complex soundscapes, overlapping events, and class imbalance, particularly in fine-grained biodiversity monitoring scenarios. In this study, we present CNNDualMamba, a new neural architecture made for multi-label anuran species classification. Our model combines three main parts: First, it uses a convolutional neural network as the backbone to efficiently extract time-frequency features. Second, it introduces a loss function that’s more suited to multi-label tasks. This function combines zero-bounded log-sum-exp and pairwise rank-based loss. Third, it designs a new dual-branch Mamba module. This module models time and frequency dependencies separately. In addition, we carry out label-aware data augmentation to solve the data imbalance problem. Experimental results on AnuraSet show that our model outperforms existing baselines and achieves an overall F1 score of 60.1%. For three subsets, our best model achieves an F1 score of 86.5%, 81.1%, and 49.8% for frequent, common, and rare classes, respectively.