<p>Sentence classification typically relies on textual information in high-resource languages. However, it is rather challenging to perform such tasks in under-resourced languages that often lack sufficient text or speech data to support such tasks. This paper examines the effectiveness of audio features derived from speech data (speech features) in the endangered tonal language Chokri for classifying sentence types using traditional machine learning and transfer learning algorithms. Results show that speech features can be effectively used for sentence classification tasks without relying on textual information. Two approaches are employed to achieve the sentence classification tasks. First, we extracted various individual and combined speech features, including Mel Frequency Cepstral Coefficients (MFCCs), Chroma, and other spectral features, as numerical data and applied the random forest algorithm for traditional machine learning. The results show that the average F1 score for MFCCs alone, MFCCs with other features, and MFCCs with Chroma is 86% each, with the highest average of 88% achieved by combining all features. These findings underscore the critical role of MFCCs in speech-based sentence classification and the varying contributions of other features. The second approach involves converting speech into spectrogram and MFCC images and applying transfer learning models like ResNet101 and InceptionResNet. This method improves classification accuracy by 10%, resulting in 97–98% accuracy. This study suggests further experimentation with various grammatical structures by demonstrating the effectiveness of speech features alone for sentence classification, thereby expanding the possibilities for language processing tasks in resource-limited scenarios.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Speech-driven sentence type classification in Chokri using traditional and transfer learning methods

  • Amalesh Gope,
  • Anusuya Pal,
  • Tulika Gogoi,
  • Sekholu Tetseo,
  • Bibek Nayan Deka,
  • Akangsha D. Nath

摘要

Sentence classification typically relies on textual information in high-resource languages. However, it is rather challenging to perform such tasks in under-resourced languages that often lack sufficient text or speech data to support such tasks. This paper examines the effectiveness of audio features derived from speech data (speech features) in the endangered tonal language Chokri for classifying sentence types using traditional machine learning and transfer learning algorithms. Results show that speech features can be effectively used for sentence classification tasks without relying on textual information. Two approaches are employed to achieve the sentence classification tasks. First, we extracted various individual and combined speech features, including Mel Frequency Cepstral Coefficients (MFCCs), Chroma, and other spectral features, as numerical data and applied the random forest algorithm for traditional machine learning. The results show that the average F1 score for MFCCs alone, MFCCs with other features, and MFCCs with Chroma is 86% each, with the highest average of 88% achieved by combining all features. These findings underscore the critical role of MFCCs in speech-based sentence classification and the varying contributions of other features. The second approach involves converting speech into spectrogram and MFCC images and applying transfer learning models like ResNet101 and InceptionResNet. This method improves classification accuracy by 10%, resulting in 97–98% accuracy. This study suggests further experimentation with various grammatical structures by demonstrating the effectiveness of speech features alone for sentence classification, thereby expanding the possibilities for language processing tasks in resource-limited scenarios.