Part-of-speech (POS) tagging is a fundamental aspect of Natural Language Processing (NLP), serving as an important preprocessing step for numerous downstream tasks such as machine translation, sentiment analysis, and information retrieval. This study explores the application of neural models, specifically Gated Recurrent Units (GRUs) and Bidirectional GRUs (Bi-GRUs), for POS tagging in Assamese, a low-resource and morphologically rich language. With manually annotated datasets, the models were trained and tested for their ability to identify the grammatical class of words. The Bi-GRU model achieved a better overall accuracy of 93.2%, while the GRU model achieved only 89%. The results reflect the superiority of bidirectional hierarchies in the use of more context information. The model limitation with ambiguous and rare tags is still an issue due to the lack of training data. This work is an extension toward developing NLP for Assamese and other low-resource languages with emphasis on the need for larger datasets and exploration into the latest structures such as transformers to enable further enhancements.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Part-of-Speech Tagging for Assamese on LDCIL Tagged Dataset Using Deep Learning

  • Rituraj Phukan,
  • Nomi Baruah,
  • Shikhar Kr. Sarma,
  • Darpanjit Konwar

摘要

Part-of-speech (POS) tagging is a fundamental aspect of Natural Language Processing (NLP), serving as an important preprocessing step for numerous downstream tasks such as machine translation, sentiment analysis, and information retrieval. This study explores the application of neural models, specifically Gated Recurrent Units (GRUs) and Bidirectional GRUs (Bi-GRUs), for POS tagging in Assamese, a low-resource and morphologically rich language. With manually annotated datasets, the models were trained and tested for their ability to identify the grammatical class of words. The Bi-GRU model achieved a better overall accuracy of 93.2%, while the GRU model achieved only 89%. The results reflect the superiority of bidirectional hierarchies in the use of more context information. The model limitation with ambiguous and rare tags is still an issue due to the lack of training data. This work is an extension toward developing NLP for Assamese and other low-resource languages with emphasis on the need for larger datasets and exploration into the latest structures such as transformers to enable further enhancements.