Natural Language Understanding (NLU) is considered a core component in implementing Dialogue Systems (DS). This paper introduces an NLU module tailored to the Tunisian Dialect (TD) that can further be integrated with Automatic Speech Recognition (ASR), a Dialogue Manager (DM), a Natural Language Generator (NLG), and a Speech Synthesis (SS) module to build a fully working DS. Designed specifically for academic and research domain, our closed-domain TUN-NLU corpus undergoes preprocessing, including scraping, automatic normalization, data augmentation, and annotation. We employed the RASA NLU model ( https://rasa.com/ ) to evaluate our corpus. The NLU model encompasses tokenization, featurization, intent detection, and entity recognition. Evaluation using standard metrics, including accuracy and F1 score, yielded promising results: 93% accuracy and 92% F1 score for intent detection, and 94% accuracy and 91% F1 score for entity recognition.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Tunisian Arabic Understanding: Resources Analysis and Evaluation

  • Fatma Zahra Besdouri,
  • Inès Zribi,
  • Lamia Hadrich Belguith

摘要

Natural Language Understanding (NLU) is considered a core component in implementing Dialogue Systems (DS). This paper introduces an NLU module tailored to the Tunisian Dialect (TD) that can further be integrated with Automatic Speech Recognition (ASR), a Dialogue Manager (DM), a Natural Language Generator (NLG), and a Speech Synthesis (SS) module to build a fully working DS. Designed specifically for academic and research domain, our closed-domain TUN-NLU corpus undergoes preprocessing, including scraping, automatic normalization, data augmentation, and annotation. We employed the RASA NLU model ( https://rasa.com/ ) to evaluate our corpus. The NLU model encompasses tokenization, featurization, intent detection, and entity recognition. Evaluation using standard metrics, including accuracy and F1 score, yielded promising results: 93% accuracy and 92% F1 score for intent detection, and 94% accuracy and 91% F1 score for entity recognition.