Named Entity Recognition in Bangla Using Conditional Random Filed (CRF)
摘要
Named Entity Recognition (NER) in the Bangla language is field of diversity towards Natural Language Processing. As Bengali is one of the top spoken languages in the world, there are so many factors and subjects to research with. This research is conducted because of the scarcity of lexical analysis in bangla language and scarcity of the well-annotated dataset. This research encompasses a comprehensive methodology, including data collection from diverse Bangla newspapers, feature engineering, and model training. The aim of this research is to contribute to the field of NLP with newer approach using diverse dataset. The dataset, meticulously curated to align with the complexities of NER, consists of almost 150,000 words and 8 unique NER tags. In the analysis phase, 8 classes were identified, and a strategic split of 20% for testing and 80% for training facilitated the evaluation of the models. Notably, the CRF model demonstrated superior performance, achieving an impressive Precision 93%, Recall 94%, and F1-Score 93.4%. Therefore, the accuracy of the CRF model comes 93.4%. The results underscore the efficacy of leveraging sequential dependencies and contextual information in the CRF model for accurate entity recognition. The discussion delves into the challenges of NER in the Bangla language, emphasizing the need for a balanced dataset and the development of a custom evaluation matrix. So, the CRF model emerged as the superior choice. This research contributes insights into the nuances of NER in Bangla, offering a tailored dataset, meticulous analysis, and model comparison. The findings emphasize the superiority of the CRF model, providing a valuable foundation for advancing NER applications in the Bangla language.