Scholarlink

<p>The performance of large language models (LLMs) for Oromo Word Sense Disambiguation (WSD) in a low-resource setting is investigated in this work. In addition to zero-shot, few-shot, and comprehensive fine-tuning paradigms, special attention is given to parameter-efficient adaptation methods such as LoRA and MoA+ROSA. MoA+ROSA maintains the same architecture by incorporating ROSA regularization and using the same LoRA backbone (since PEFT does not support mixture-of-adapters).The tests use WordPiece tokenization and [CLS] token embeddings to capture semantic differences across 60 senses in a controlled dataset, utilizing BERT-base-multilingual-cased. A comparative investigation reveals that low-rank adaptation techniques perform poorly in situations of extreme data sparsity, but few-shot and full fine-tuning attain high accuracy and reliable predictions, efficiently improving lexical representations for precise sense identification. Transferring knowledge from high-resource languages to Oromo, a morphologically rich African language, can result in alignment patterns and possible performance loss, according to cross-lingual transfer analyses. Diagnostic indicators, such as variance, bias, and per-class degradation (RelDrop), shed light on the inconsistent performance across senses and the dependability of the model. The tiny dataset size, the poor performance of parameter-efficient adaptations, and the paucity of ablation investigations are some of the limitations that limit our comprehension of component-level contributions. When taken as a whole, the results improve low-resource NLP by showing how contextual embeddings, adaption techniques, and fine-tuning work together resolve sense ambiguity in a low-resource, morphologically rich African language.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Evaluating adaptation strategies for low resource word sense disambiguation in Oromo using the BERT base multilingual cased model

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月2日

查看全文
  • Liyachew Edeti,
  • Million Meshesha,
  • Feda Negesse

摘要

The performance of large language models (LLMs) for Oromo Word Sense Disambiguation (WSD) in a low-resource setting is investigated in this work. In addition to zero-shot, few-shot, and comprehensive fine-tuning paradigms, special attention is given to parameter-efficient adaptation methods such as LoRA and MoA+ROSA. MoA+ROSA maintains the same architecture by incorporating ROSA regularization and using the same LoRA backbone (since PEFT does not support mixture-of-adapters).The tests use WordPiece tokenization and [CLS] token embeddings to capture semantic differences across 60 senses in a controlled dataset, utilizing BERT-base-multilingual-cased. A comparative investigation reveals that low-rank adaptation techniques perform poorly in situations of extreme data sparsity, but few-shot and full fine-tuning attain high accuracy and reliable predictions, efficiently improving lexical representations for precise sense identification. Transferring knowledge from high-resource languages to Oromo, a morphologically rich African language, can result in alignment patterns and possible performance loss, according to cross-lingual transfer analyses. Diagnostic indicators, such as variance, bias, and per-class degradation (RelDrop), shed light on the inconsistent performance across senses and the dependability of the model. The tiny dataset size, the poor performance of parameter-efficient adaptations, and the paucity of ablation investigations are some of the limitations that limit our comprehension of component-level contributions. When taken as a whole, the results improve low-resource NLP by showing how contextual embeddings, adaption techniques, and fine-tuning work together resolve sense ambiguity in a low-resource, morphologically rich African language.