Multilingual speech recognition initiative for African languages

This paper summarizes speech recognition systems for African languages. More precisely, we propose different approaches that address the low-resource property of these languages. For both monolingual and multilingual systems, our methods rely on self-supervised pre-trained models for multiple languages. We tested our method on 7 African languages and dialects: Amharic, Darija, Fongbe, Sudanese, Swahili, Wolof, and Yoruba. We first trained monolingual models that were used as baselines and then proposed proof-of-concepts for systems that handle multiple languages. Our multilingual contribution was based on three approaches. (a) We trained a single model by concatenating the multilingual corpora with no specific data processing. Because of confusions between the different languages, this “naïve approach” led to a grapheme overlapping within the transcriptions. (b) To avoid this, we experimented a second multilingual model by feeding a one-hot encoder vector to the speech features on training. For this purpose, a language identification model is required on inference. (c) Finally, we proposed a last model that can predict the spoken language using language-specific tokens added to the text transcription. The aim of this method is to avoid the necessity to load a language identification on each inference which can be time- and resource-consuming. We also investigated the impact of lexical ambiguity by removing diacritics from text for the concerned languages.