The effectiveness of cross-lingual transfer in Transformer models is critical for applying NLP to low-resource languages, yet its practical boundaries remain underexplored. This work investigates this phenomenon through a case study on sensitive information classification in Portuguese, explicitly mapping the limits and surprises of cross-lingual adaptation. We systematically compare four BERT variants: a specialized Portuguese model (BERTimbau), a multilingual model (mBERT), and two monolingual models from distant languages (English and Chinese). Our findings reveal that while the specialized model’s superiority (99.49% F1-score) highlights the performance limit of fine-tuning across typologically distant corpora, the results also uncover a surprising degree of architectural adaptability. The multilingual model is a strong baseline (97.95% F1-score), and even the model pre-trained on Chinese achieves high functional performance (89.14% F1-score), demonstrating that much of the knowledge learned during pre-training is language-agnostic and can be effectively repurposed. These results offer a clear quantification of the trade-off between specialization and adaptability, providing valuable insights for implementing classification systems in data-scarce contexts.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Limits and Surprises of Cross-Lingual BERT Under Data Scarcity: A Portuguese Case Study in Sensitive-Text Classification

  • Wellington Fernandes Silvano,
  • Maurício Konrath,
  • Ricardo F. Custódio

摘要

The effectiveness of cross-lingual transfer in Transformer models is critical for applying NLP to low-resource languages, yet its practical boundaries remain underexplored. This work investigates this phenomenon through a case study on sensitive information classification in Portuguese, explicitly mapping the limits and surprises of cross-lingual adaptation. We systematically compare four BERT variants: a specialized Portuguese model (BERTimbau), a multilingual model (mBERT), and two monolingual models from distant languages (English and Chinese). Our findings reveal that while the specialized model’s superiority (99.49% F1-score) highlights the performance limit of fine-tuning across typologically distant corpora, the results also uncover a surprising degree of architectural adaptability. The multilingual model is a strong baseline (97.95% F1-score), and even the model pre-trained on Chinese achieves high functional performance (89.14% F1-score), demonstrating that much of the knowledge learned during pre-training is language-agnostic and can be effectively repurposed. These results offer a clear quantification of the trade-off between specialization and adaptability, providing valuable insights for implementing classification systems in data-scarce contexts.