Advancing Bird Species Classification: A Fusion of Audio and Image Data
摘要
Automated classification of bird species is crucial for large-scale environmental monitoring, providing valuable insights into temporal and spatial changes in ecosystems. Previous studies have primarily focused on using either acoustic or visual data for bird species recognition. However, few studies have explored the simultaneous use of both acoustic and visual data to improve classification performance. In this study, we propose a dual branch network based on pre-trained models to enhance bird species classification by integrating acoustic and visual information. Specifically, ResNet50 is used for visual data, while CNN14 is employed for acoustic data. The extracted feature embeddings are then fused, and attention mechanisms are applied to further improve classification performance. Experimental results demonstrate that our proposed model achieves significantly higher accuracy compared to using audio or image data alone. The best-performing model achieved an accuracy of 96.44%, precision of 96.62%, recall of 94.30%, and F1-score of 95.01%. This study highlights the potential of combining acoustic and visual data for bird species classification and suggests that attention mechanisms can further enhance model performance.