In this work, we propose a system for the automatic recognition of linguistic varieties spoken in Sardinia, with the aim of filling the current gap in NLP systems for minority languages. Starting from a dataset consisting of over 29,000 transcripts from speakers from 203 Sardinian locations, we built two variants: a multi-class dataset with five language labels (Campidanese, Logudorese, Nuorese, Sassarese, Gallurese) and a binary dataset containing only the Sardinian and Sassarese varieties; both datasets were used for fine-tuning BERT. The results show high classification accuracy, exceeding 96% in the multi-class case and 97% in the binary case. In the latter scenario, our model outperformed GlotLID, a state-of-the-art reference system. This work demonstrates that advanced NLP techniques can also be effectively applied to poorly represented linguistic contexts, contributing to the enhancement of cultural heritage. The system developed can be easily extended to other Italian and European minority languages, opening up new perspectives for the documentation and protection of linguistic diversity.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Preserving Sardinian Linguistic Diversity: An Automatic Language Identifier Approach

  • Salvatore Mario Carta,
  • Gianni Fenu,
  • Alessandro Giuliani,
  • Marco Manolo Manca,
  • Mirko Marras,
  • Leonardo Piano,
  • Alessandro Sebastian Podda

摘要

In this work, we propose a system for the automatic recognition of linguistic varieties spoken in Sardinia, with the aim of filling the current gap in NLP systems for minority languages. Starting from a dataset consisting of over 29,000 transcripts from speakers from 203 Sardinian locations, we built two variants: a multi-class dataset with five language labels (Campidanese, Logudorese, Nuorese, Sassarese, Gallurese) and a binary dataset containing only the Sardinian and Sassarese varieties; both datasets were used for fine-tuning BERT. The results show high classification accuracy, exceeding 96% in the multi-class case and 97% in the binary case. In the latter scenario, our model outperformed GlotLID, a state-of-the-art reference system. This work demonstrates that advanced NLP techniques can also be effectively applied to poorly represented linguistic contexts, contributing to the enhancement of cultural heritage. The system developed can be easily extended to other Italian and European minority languages, opening up new perspectives for the documentation and protection of linguistic diversity.