Deep Learning for Predicting the Next Word in Bilingual Social Media Texts

This paper presents a novel architecture for predicting the next word in bilingual Punjabi-English (BPE) social media texts. The goal is to enhance the performance and accuracy of next-word prediction in multilingual environments. Our proposed model, called NWP-CB (Next-Word Prediction using CNN and BiLSTM), incorporates two powerful deep learning techniques: Convolutional Neural Network (CNN) and Bidirectional Long Short-Term Memory (BiLSTM). We collected a large dataset from Twitter and WhatsApp, consisting of approximately 500,000 tweets and 600,000 WhatsApp messages in Punjabi-English. After careful preprocessing, we trained and evaluated our model using this dataset. By leveraging the CNN to extract features that capture local dependencies and the BiLSTM to capture long-range dependencies, our model offers superior performance compared to five other deep learning models. We achieved a high training accuracy of 0.9367 and a low training loss of 0.2765 within just 270.80 min. To enhance the efficiency and scalability of our NWP-CB model, we increased the number of neurons per layer and removed dropout layers. Our findings demonstrate the effectiveness of our proposed architecture in advancing natural language processing techniques for multilingual contexts. The practical implications of this work include the development of predictive models that can be applied across diverse linguistic domains, facilitated by a tool developed to use the trained model.