Temperature is All You Need: Approximating Human Mathematics Hint Efficacy with LLMs
摘要
This study explores the potential of large language models (LLMs) to approximate human performance in mathematics under varying levels of instructional support. We compare responses from human participants to those from GPT-3.5 and Llama 3 across six experimental conditions, ranging from 0% (no hint) to 100% (fully worked solution), with intermediate levels (20%, 40%, 60%, 80%) representing partial hints. Varying the temperature parameter, we found that setting it to 2 yielded the highest correlation with human accuracy–92% for GPT-3.5 and 96% for Llama 3–along with similar exponential performance curves. In contrast, other temperature settings led to flatter or logarithmic learning curves that plateaued at 60% hinting. Our findings suggest that partial worked solutions of 80% offer significant instructional value and that higher temperatures in LLMs may better simulate human learning trajectories. This enables rapid, cost-effective prototyping of tutoring interventions prior to student deployment.