<p>A key task in natural language processing is word sense disambiguation (WSD), which attempts to determine the accurate meaning of ambiguous words based on their context. While transformer-based designs have achieved significant results in high-resource languages, WSD for low-resource languages such as Oromo remains hard due to inadequate annotated corpora and lexical resources. Contextual representation learning has been greatly enhanced by recent advancements in transformer-based language models, allowing for more reliable disambiguation in situations with limited input. This study uses a manually created dataset from the Oromo–English Dictionary to examine the efficacy of transformer-based models for lexical-sample WSD in Oromo. The dataset contains sentences annotated by two native speakers, attaining an inter-annotator agreement of 0.82, indicating good annotation reliability. The dataset was filtered for experimental usage following preprocessing, normalization, and elimination of noisy cases. 472 training sentences, 71 validation sentences (15%), and 140 test sentences made up the final dataset. The dataset has a highly unbalanced long-tail distribution and encompasses 43 sense classes. BERT-base-cased gets the best performance with an accuracy of 0.862 and a macro-F1 score of 0.2897, according to an experimental evaluation of transformer-based models, including BERT, RoBERTa, DistilBERT, Davlan/afro-xlmr-base, and multilingual variations. Significant differences between models with χ<sup>2</sup> = 34.03 and <i>p</i> = 4.0 × 10<sup>−1</sup> are confirmed by statistical analysis using the Friedman test. BERT-base-cased performs much better than most transformer variations and classical baselines, according to post-hoc Wilcoxon signed-rank tests. These results show that contextual transformer representations are quite successful for low-resource WSD, although there is still a significant class imbalance that limits performance.</p>

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Low resource word sense disambiguation in Oromo with fine tuned small transformers

  • Liyachew Edeti,
  • Million Meshesha,
  • Feda Negesse

摘要

A key task in natural language processing is word sense disambiguation (WSD), which attempts to determine the accurate meaning of ambiguous words based on their context. While transformer-based designs have achieved significant results in high-resource languages, WSD for low-resource languages such as Oromo remains hard due to inadequate annotated corpora and lexical resources. Contextual representation learning has been greatly enhanced by recent advancements in transformer-based language models, allowing for more reliable disambiguation in situations with limited input. This study uses a manually created dataset from the Oromo–English Dictionary to examine the efficacy of transformer-based models for lexical-sample WSD in Oromo. The dataset contains sentences annotated by two native speakers, attaining an inter-annotator agreement of 0.82, indicating good annotation reliability. The dataset was filtered for experimental usage following preprocessing, normalization, and elimination of noisy cases. 472 training sentences, 71 validation sentences (15%), and 140 test sentences made up the final dataset. The dataset has a highly unbalanced long-tail distribution and encompasses 43 sense classes. BERT-base-cased gets the best performance with an accuracy of 0.862 and a macro-F1 score of 0.2897, according to an experimental evaluation of transformer-based models, including BERT, RoBERTa, DistilBERT, Davlan/afro-xlmr-base, and multilingual variations. Significant differences between models with χ2 = 34.03 and p = 4.0 × 10−1 are confirmed by statistical analysis using the Friedman test. BERT-base-cased performs much better than most transformer variations and classical baselines, according to post-hoc Wilcoxon signed-rank tests. These results show that contextual transformer representations are quite successful for low-resource WSD, although there is still a significant class imbalance that limits performance.