Language Identification Using Multinomial Naive Bayes Technique

Language detection is a significant effort in natural language processing (NLP) and has various applications such as machine translation, text summarization, and sentiment analysis. In this paper, we propose using a Multinomial Naive Bayes (MNB) algorithm for the task of language detection. MNB is a widely used algorithm in NLP and is effective in various text classification tasks, including language detection. In this research, we propose using MNB for the task of language detection. We used a dataset of texts written in different languages to train the algorithm. The dataset was preprocessed to extract features and remove halts. The MNB algorithm was implemented using the scikit-learn library in Python. The algorithm was first trained, and the set used for it was termed as training set and then was tested on the testing set. Using the accuracy, the algorithm’s performance was estimated. This paper is organized into five sections: Sect. 1 is introduction, Section 2 is literature review and research gap, Sect. 3 includes the implementations and discussion, Sect. 4 consists of the main resultant part, and Sect. 5 concludes with the future.