In this paper we present a short study on the user perspective of the capability of LLM-based AI to verbalize mathematical content, i.e. transcribe symbolic notation and formulas into natural spoken language. For the selected base of mathematical expressions, we run a series of experiments and analyze the results of verbalization obtained by prompting LLM in terms of repeatability and precision. As a reference we use the output of the Equation Wizard – the efficient rule-based verbalization tool designed and developed by the authors in previous works. Our experiments are performed with the use of ChatGPT 3.5 – a popular, free-of charge LLM that is frequently and eagerly used by students. We demonstrate the inconsistency and unrepeatability of LLM output based on repeated verbalization requests, as well as showcase imperfect verbalization.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Capability of LLM-Based AI to Verbalize Math: A User Perspective Case Study

  • Agnieszka Bier,
  • Zdzislaw Sroczynski

摘要

In this paper we present a short study on the user perspective of the capability of LLM-based AI to verbalize mathematical content, i.e. transcribe symbolic notation and formulas into natural spoken language. For the selected base of mathematical expressions, we run a series of experiments and analyze the results of verbalization obtained by prompting LLM in terms of repeatability and precision. As a reference we use the output of the Equation Wizard – the efficient rule-based verbalization tool designed and developed by the authors in previous works. Our experiments are performed with the use of ChatGPT 3.5 – a popular, free-of charge LLM that is frequently and eagerly used by students. We demonstrate the inconsistency and unrepeatability of LLM output based on repeated verbalization requests, as well as showcase imperfect verbalization.