Assessment of ChatGPT in extracting Arabic morphology: active and passive participles

ChatGPT is a language model that has been developed by Open AI. It has the ability to answer any question raised by a participant in a form of conservation. ChatGPT takes advantages of the effective transformers architecture to generate text discussions that are relevant and perceptive. Although ChatGPT was initially trained on English text data, it has also been fine-tuned on other languages including Spanish, French, Chinese, German and Arabic. ChatGPT has been employed in different NLP domains for English language, but specifically it is still slightly limited in languages other than English. In this research, we assess ChatGPT in extracting active participle (ism al-fa’il) and passive participle (ism al-maf’uul) from Arabic verbs of base forms (alfilu alsahih) and verbs with vowels (alfilu almutal) to help the non-native speakers who find it difficult to extract these participles and try to use ChaTGPT as a learning tool and get inaccurate responses. We experiment ChatGPT’s performance in zero-shot learning then with a wide range of prompt templates as well as few-shot learning. The results show a greet enhancement in the ChatGPT performance when tuning the rule-based prompts with k-shot examples especially for passive participle extraction. ChatGPT performed well in extracting participles from verbs without vowels, but after training with k-shot samples, it improved by 6% for active participles and 25% for passive participles. Additionally, extracting passive participles for verbs with vowels significantly improves performance compared to extracting active participles, especially the case when the vowels exist at two positions, which results in a 100% improvement. The performance of ChatGPT improved in extracting active participle for verbs with no vowels by 6% and passive participles by 25%.