Scholarlink

In this paper, we present a comprehensive literature review on the capabilities of large language models (LLMs) in answering AI-related questions in Bengali. We first reviewed the various datasets available in Bengali, followed by an analysis of multilingual LLMs capable of generating responses in the language. To assess the performance of these models, we curated a list of 50 AI questions in Bengali. With input from AI experts, these questions were categorized into three levels of complexity: simple, moderate, and complex. We then posed these questions to leading LLMs, including ChatGPT, Gemini, and Claude, and evaluated their responses based on two criteria: correctness and linguistic formation. Domain experts carried out these evaluations. Our study reveals notable areas for improvement in the performance of LLMs, especially in providing accurate and well-formed responses in Bengali. Enhancing the capability of LLMs in a major Indian language like Bengali is crucial for advancing AI education in the region.

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

An Empirical Evaluation for LLMs Performance on AI Question and Answer in Bengali

  • 会议论文

  • 发表日期：

    2026年

查看全文
  • Susmita Koner,
  • Saptarsi Goswami,
  • Amlan Chakrabarti,
  • Basabi Chakraborty

摘要

In this paper, we present a comprehensive literature review on the capabilities of large language models (LLMs) in answering AI-related questions in Bengali. We first reviewed the various datasets available in Bengali, followed by an analysis of multilingual LLMs capable of generating responses in the language. To assess the performance of these models, we curated a list of 50 AI questions in Bengali. With input from AI experts, these questions were categorized into three levels of complexity: simple, moderate, and complex. We then posed these questions to leading LLMs, including ChatGPT, Gemini, and Claude, and evaluated their responses based on two criteria: correctness and linguistic formation. Domain experts carried out these evaluations. Our study reveals notable areas for improvement in the performance of LLMs, especially in providing accurate and well-formed responses in Bengali. Enhancing the capability of LLMs in a major Indian language like Bengali is crucial for advancing AI education in the region.