In response to the urgent need for accelerated decision-making during health crises, this research underscores the pivotal role of keyword spotting (KWS) within continuous multilingual speech frameworks. Our methodology encompasses four critical phases: Dataset Selection, Feature Extraction, Model Training, and Posterior Handling. We assembled a diverse dataset of 42 recordings in English, French, and Arabic, totaling \( \approx 35\) minutes. MFCCs were employed for feature extraction due to their alignment with human auditory perception. For model training, we evaluated two Convolutional Neural Network (CNN) architectures, ResNet-18 and ResNet-152, comparing their performance in recognizing keywords across multilingual contexts. The dataset was preprocessed to include MFCCs and contextual embeddings for predefined keywords using Multilingual BERT, creating an integrated representation for model input. Experimental results focused on accuracy, loss, and F1 score demonstrate that ResNet-18 achieved superior performance with 90.26% accuracy and an F1 score of 95.75%  outperforming ResNet-152, which attained 88.78% accuracy and an F1 score of 88.79%. These results highlight ResNet-18’s effectiveness in multilingual KWS tasks, making it a valuable tool for rapid and accurate decision-making during health crises.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

ResNet-Based Pandemic Keyword Spotting in Continuous Multilingual Speech: A Study in UNESCO’s Audio Messages for Rapid Health Response

  • Samawel Jaballi,
  • Manar Joundy Hazar,
  • Salah Zrigui,
  • Henri Nicolas,
  • Mounir Zrigui

摘要

In response to the urgent need for accelerated decision-making during health crises, this research underscores the pivotal role of keyword spotting (KWS) within continuous multilingual speech frameworks. Our methodology encompasses four critical phases: Dataset Selection, Feature Extraction, Model Training, and Posterior Handling. We assembled a diverse dataset of 42 recordings in English, French, and Arabic, totaling \( \approx 35\) minutes. MFCCs were employed for feature extraction due to their alignment with human auditory perception. For model training, we evaluated two Convolutional Neural Network (CNN) architectures, ResNet-18 and ResNet-152, comparing their performance in recognizing keywords across multilingual contexts. The dataset was preprocessed to include MFCCs and contextual embeddings for predefined keywords using Multilingual BERT, creating an integrated representation for model input. Experimental results focused on accuracy, loss, and F1 score demonstrate that ResNet-18 achieved superior performance with 90.26% accuracy and an F1 score of 95.75%  outperforming ResNet-152, which attained 88.78% accuracy and an F1 score of 88.79%. These results highlight ResNet-18’s effectiveness in multilingual KWS tasks, making it a valuable tool for rapid and accurate decision-making during health crises.