Using Pre-trained Models for Code-Switched Speech Recognition

In various regions of the world, people tend to use a mix of multiple languages in day-to-day communication. The multilingual ASR system must recognize sentences/words that switch between languages seamlessly. Many of the early approaches required in-depth linguistic knowledge of the languages to function. This issue has been addressed using deep learning methods in combination with Hidden Markov Models, such as in HMM-DNN. However, the HMM-DNN model requires forced segmentation alignment of data. End-to-end models based on self-attention, such as Listen, Attend, and Spell, and Transformers, along with Connectionist Temporal Classification (CTC) models, are easier to train and do not require bootstrapping or time alignments to train. Many existing bilingual and multilingual ASR needs Language Identification (LID) model followed by monolingual ASR. But, the overall Word Error Rate (WER) of existing state-of-the-art systems is considerably high. In this work, we would like to analyze the use of transfer learning to improve the performance of code-switched ASR models. XLS-R, a transformer model pre-trained on unlabeled audio sequences, has been used in the proposed system by fine-tuning it on a Hindi-English code-switched dataset. It is observed that this approach provides better performance in recognizing multilingual ASR. Furthermore, the design of the system is simple and does not require language identification models. We have observed that the performance is higher for limited set of data.