Code-Mixed Language Understanding Using BiLSTM-BERT Multi-attention Fusion Mechanism

Code-mixed language, characterized by the seamless blending of multiple languages, presents a formidable challenge for natural language understanding systems. In our work, we have propose a novel approach to address the complexities of code-mixed text comprehension by combining BiLSTM-BERT models with a multi-attention fusion mechanism. This paper proposes a novel approach for Code-mixed language joint intent classification and slot filling (JIC-SF) using a BiLSTM-BERT multi-attention model. The proposed model employs multi-attention mechanisms, including self-attention and cross-attention, to dynamically weigh the importance of different parts of the input text for JIC-SF. We evaluated our model on benchmark datasets: ATIS, SNIPs and Hind-English Code-mixed (HiEn-CMD) datasets. The results demonstrate that our approach outperforms the state-of-the-art models on benchmark datasets. Specifically, our model achieved an intent accuracy of 97.87% and a slot F1-score of 95.97% on the ATIS dataset, an intent accuracy of 98.86% and a slot F1-score of 96.25% on the SNIPs dataset, and intent accuracy of 84.24% and a slot F1-score of 82.68% on the HiEn-CMD dataset. Our proposed BiLSTM-BERT multi-attention model for JIC-SF provides a promising solution to improve the accuracy and efficiency of natural language understanding (NLU) systems, which can benefit various applications such as virtual assistants, chatbots, and customer service systems.