MbAbI: A Benchmark Dataset for Malayalam Text Understanding and Reasoning

This paper proposes a Malayalam Dataset containing 40000 instances, intended to assist the researchers working on Question Answering systems. This data set is the first of its kind. The Facebook bAbI dataset was translated to create this Malayalam bAbI (or, MbAbI) dataset. It comprises 20 different tasks of varying complexity. The tasks include simple supporting facts to complex path finding tasks. We have machine translated these tasks from English to Malayalam and tested the baseline QA models on these datasets. We have obtained state-of-the-art results with MbAbI dataset using deep learning and transformer models. In order to benefit the low-resource Malayalam research community we have made this dataset publicly available.