An Arabic Chatbot Leveraging Encoder-Decoder Architecture Enhanced with BERT

This paper introduces a new method for developing an Arabic chatbot, utilizing the encoder-decoder architecture enriched with BERT embeddings. Our distinct dataset, constructed manually, aids the model in comprehending intricate questions and prompts, thereby generating coherent and contextually accurate responses in Arabic. The dataset comprises 81,659 manually created conversation pairs. Our model successfully delivered the anticipated answers. We employed a model with a warm-start using the BERT2BERT encoder and decoder. It achieved a BLEU score of 3.52 and a PPL of 36.3.