Scholarlink

<p>In this research, we assessed four publicly available large-language models(LLM): ChatGPT-4o, Claude 3.5 Sonnet, DeepSeek V2, and Gemini 1.5 Pro, to study a curated set of 10 math questions, varying in fields such as algebra, arithmetic, trigonometry, geometry, and statistics, categorised by topic and different levels of complexity. We used Text analysis, Cosine similarity, Jaccard similarity and a Column chart of word-count to compare the results given by each model. To analyse the performance thoroughly, we applied Analysis Of Variance (ANOVA) for word count, Fleschscore, Polarity and Sensitivity analysis on the results to test accuracy, clarity of explanation, range of problems supported, ease of platform navigation and possible errors made by the chatbots. Our findings show slight variation in the performance among the chatbots. This research also highlights the limitations of each model providing insights to make optimum use of these chatbots in solving mathematical problems.</p>

错误：搜索内容不能为空，请输入英文关键词
错误：关键词超出字数限制，请精简
高级检索

Comparative analysis of AI chatbot mathematical performance

  • 期刊论文

  • 出版模式：

    开放获取

  • 发表日期：

    2026年6月17日

查看全文
  • Nighna Bharwani,
  • Shrinidhi Rao,
  • Manisha Kankarej

摘要

In this research, we assessed four publicly available large-language models(LLM): ChatGPT-4o, Claude 3.5 Sonnet, DeepSeek V2, and Gemini 1.5 Pro, to study a curated set of 10 math questions, varying in fields such as algebra, arithmetic, trigonometry, geometry, and statistics, categorised by topic and different levels of complexity. We used Text analysis, Cosine similarity, Jaccard similarity and a Column chart of word-count to compare the results given by each model. To analyse the performance thoroughly, we applied Analysis Of Variance (ANOVA) for word count, Fleschscore, Polarity and Sensitivity analysis on the results to test accuracy, clarity of explanation, range of problems supported, ease of platform navigation and possible errors made by the chatbots. Our findings show slight variation in the performance among the chatbots. This research also highlights the limitations of each model providing insights to make optimum use of these chatbots in solving mathematical problems.