This paper introduces MAGENTA+, an enhanced dataset and framework for detecting Arabic Machine-Generated Text (MGT) across multiple domains. As Large Language Models (LLMs) advance, distinguishing between human-written and machine-generated text becomes crucial, especially for less-resourced languages like Arabic. MAGENTA+ expands previous work by adding a new domain (Twitter), increasing dataset size by 33%, and improving generated text quality and diversity. We conduct experiments using state-of-the-art transformer models (AraBERT) and classical machine learning techniques, addressing overfitting and model robustness. Our findings show ensemble methods effectively capture diverse aspects of text classification, achieving a 93% macro F1-score. We provide insights into domain-specific performance and challenges in detecting high-quality Arabic MGT.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

MAGENTA+: Detection of Arabic Machine-Generated Text Across Domains

  • Ayoub Houmaine,
  • Paolo Rosso

摘要

This paper introduces MAGENTA+, an enhanced dataset and framework for detecting Arabic Machine-Generated Text (MGT) across multiple domains. As Large Language Models (LLMs) advance, distinguishing between human-written and machine-generated text becomes crucial, especially for less-resourced languages like Arabic. MAGENTA+ expands previous work by adding a new domain (Twitter), increasing dataset size by 33%, and improving generated text quality and diversity. We conduct experiments using state-of-the-art transformer models (AraBERT) and classical machine learning techniques, addressing overfitting and model robustness. Our findings show ensemble methods effectively capture diverse aspects of text classification, achieving a 93% macro F1-score. We provide insights into domain-specific performance and challenges in detecting high-quality Arabic MGT.