AI ChatBots’ solutions to mathematical problems in interactive e-textbooks: Affordances and constraints from the eyes of students and teachers

One of the aims of the present study was to reveal and compare the performance of ChatGPT versions (GPT-4o, GPT-4, and GPT-3.5), MathGPT, and Gemini in solving 390 mathematical problems in interactive mathematics e-textbooks across various dimensions. The other aim was to identify the affordances and constraints of ChatGPT through the instrumental approach. Both quantitative and qualitative approaches were used. The participants were 160 high school students and 80 mathematics teachers. Data were collected through the evaluation forms, view forms, and interviews with students and teachers based on ChatGPT’s solutions to the mathematical problems. The findings showed that the success rates of GPT-4o and GPT-4 were close to each other, with a slightly higher success rate of GPT-4o. This was followed by MathGPT and GPT-3.5. Gemini has the lowest success rate among the AI chatbots. Depending on the complexity of the mathematical problems, a statistically significant difference between the number of correct and incorrect solutions was found in all ChatGPT versions but not in MathGPT and Gemini. Furthermore, teachers and students referred to explanatory and detailed aspects of the solutions, learning without a teacher, getting solutions directly and quickly, and learning support as affordances of ChatGPT. On the other hand, the participants also acknowledged the constraints of ChatGPT while being aware of its affordances. Based on students’ and teachers’ expressions in the view forms and interviews, the mean scores they provided in the evaluation forms, and ChatGPT’s high performance in solving mathematical problems, it is suggested that ChatGPT could be a useful tool for students’ individual mathematics learning process.