The presence of diacritics plays a crucial role in representing the meaning and pronunciation of Arabic words and sentences accurately. Over the years, researchers have dedicated considerable efforts to enhance automated diacritization systems. However, to our knowledge, Bidirectional Encoder Representations from Transformers (BERT) models were not yet investigated for the full diacritization approach. This study introduces a novel approach that utilizes transfer learning and BERT for diacritizing Arabic sentences. To show the effectiveness of the proposed approach, this research paper utilizes the Arabic Diacritization (AD) benchmark dataset to compare the constructed BERT model with the most recent research using error metrics. The research findings demonstrate the significance of the proposed approach in improving the accuracy of diacritization. The trained model outperformed all previous techniques and achieved state-of-the-art (SOTA) results in all error metrics. The resulting syntactic DER and WER are 1.14% and 3.35%, respectively. In morphological diacritization, the best results showed a DER of 0.92% and a WER of 1.91%. These results represent a significant relative error reduction of 35% on average compared to previous research.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Context-Aware Arabic Diacritization Using Transformers

  • Ruba Kharsa,
  • Ashraf Elnagar

摘要

The presence of diacritics plays a crucial role in representing the meaning and pronunciation of Arabic words and sentences accurately. Over the years, researchers have dedicated considerable efforts to enhance automated diacritization systems. However, to our knowledge, Bidirectional Encoder Representations from Transformers (BERT) models were not yet investigated for the full diacritization approach. This study introduces a novel approach that utilizes transfer learning and BERT for diacritizing Arabic sentences. To show the effectiveness of the proposed approach, this research paper utilizes the Arabic Diacritization (AD) benchmark dataset to compare the constructed BERT model with the most recent research using error metrics. The research findings demonstrate the significance of the proposed approach in improving the accuracy of diacritization. The trained model outperformed all previous techniques and achieved state-of-the-art (SOTA) results in all error metrics. The resulting syntactic DER and WER are 1.14% and 3.35%, respectively. In morphological diacritization, the best results showed a DER of 0.92% and a WER of 1.91%. These results represent a significant relative error reduction of 35% on average compared to previous research.