The evolution of technology has increased the demands of Intelligent Homes worldwide. This paper discusses customization of the Smart Home Automation systems for Indian local languages. Command identification from the audio signals is challenging. Text processing is an efficient and simpler alternative to it. Conversion of Speech to Text is performed using the approach of Automatic Speech Recognition (ASR). A comparative study is performed among the ASR models Wav2Vec2, Wav2Vec2-BERT and HuBERT. The work is targeted to fine-tune these models for Hindi and Marathi languages having low-resource datasets and to explore the best suitable self-supervised state-of-the-art model to enhance the regional user’s experience. Connectionist Temporal Classification (CTC) is used in conjunction with the ASR model for the appropriate alignment of the text characters corresponding to the input speech. The generated output transcriptions matched well with the spoken commands. The comparative analysis of the resulting fine-tuned models is performed based on the Word Error Rate (WER) metric. Their efficiency has been tested by capturing real-time voice commands given by numerous users. The best performance for both Hindi and Marathi languages is obtained from the “HuBERT” model with an WER of 0.569 and 0.526, respectively. This approach can be extended to finetune the ASR models for many other Indian dialects.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Automatic Speech Recognition for Smart Home Automation Systems in Vernacular Indian Dialects

  • Shatabdi Sankhari,
  • Sudhanshu Ramteke,
  • Janhavi Ghuge,
  • Yash Tijare,
  • Priyanshu Verma,
  • Himanshu Bankar,
  • Ashwin G. Kothari

摘要

The evolution of technology has increased the demands of Intelligent Homes worldwide. This paper discusses customization of the Smart Home Automation systems for Indian local languages. Command identification from the audio signals is challenging. Text processing is an efficient and simpler alternative to it. Conversion of Speech to Text is performed using the approach of Automatic Speech Recognition (ASR). A comparative study is performed among the ASR models Wav2Vec2, Wav2Vec2-BERT and HuBERT. The work is targeted to fine-tune these models for Hindi and Marathi languages having low-resource datasets and to explore the best suitable self-supervised state-of-the-art model to enhance the regional user’s experience. Connectionist Temporal Classification (CTC) is used in conjunction with the ASR model for the appropriate alignment of the text characters corresponding to the input speech. The generated output transcriptions matched well with the spoken commands. The comparative analysis of the resulting fine-tuned models is performed based on the Word Error Rate (WER) metric. Their efficiency has been tested by capturing real-time voice commands given by numerous users. The best performance for both Hindi and Marathi languages is obtained from the “HuBERT” model with an WER of 0.569 and 0.526, respectively. This approach can be extended to finetune the ASR models for many other Indian dialects.