Earlier this year, Apple ignited controversy by publishing “The Illusion of Thinking,” prompting heated debate within the AI community. Critics seized upon the findings as conclusive evidence that Large Reasoning Models (LRMs) lack genuine reasoning capabilities, branding them as mere stochastic parrots. Meanwhile, defenders—spearheaded by Lawsen et al. (2025)—fired back, condemning the experimental setup as flawed and the conclusions overstated. We clarify this debate by replicating and refining the original study’s most contentious benchmarks: Towers of Hanoi, River Crossing, Blocks World and Checker Jumping. By introducing incremental stepwise prompting and agentic collaborative dialogue, we show that previously reported failures solving these puzzles were not purely result of output constraints, but also partly a result of cognition limitations: LRMs still stumble when complexity rises moderately (around 8 disks in Towers of Hanoi). Moreover, the River Crossing results initially heralded as catastrophic failures turn out to hinge upon testing unsolvable configurations. Once we limit tests strictly to solvable problems—LRMs effortlessly solve large instances involving over 100 agent pairs. Our findings ultimately defy simplistic narratives: today’s LRMs are stochastic, RL-tuned searchers in a discrete state space we barely understand. Real progress in symbolic, long-horizon reasoning demands mapping that terrain through fine-grained ablations like those introduced here.

错误:搜索内容不能为空,请输入英文关键词
错误:关键词超出字数限制,请精简
高级检索

Rethinking the Illusion of Thinking

  • Iñaki Dellibarda Varela,
  • Pablo Romero-Sorozabal,
  • Eduardo Rocon,
  • Manuel Cebrian

摘要

Earlier this year, Apple ignited controversy by publishing “The Illusion of Thinking,” prompting heated debate within the AI community. Critics seized upon the findings as conclusive evidence that Large Reasoning Models (LRMs) lack genuine reasoning capabilities, branding them as mere stochastic parrots. Meanwhile, defenders—spearheaded by Lawsen et al. (2025)—fired back, condemning the experimental setup as flawed and the conclusions overstated. We clarify this debate by replicating and refining the original study’s most contentious benchmarks: Towers of Hanoi, River Crossing, Blocks World and Checker Jumping. By introducing incremental stepwise prompting and agentic collaborative dialogue, we show that previously reported failures solving these puzzles were not purely result of output constraints, but also partly a result of cognition limitations: LRMs still stumble when complexity rises moderately (around 8 disks in Towers of Hanoi). Moreover, the River Crossing results initially heralded as catastrophic failures turn out to hinge upon testing unsolvable configurations. Once we limit tests strictly to solvable problems—LRMs effortlessly solve large instances involving over 100 agent pairs. Our findings ultimately defy simplistic narratives: today’s LRMs are stochastic, RL-tuned searchers in a discrete state space we barely understand. Real progress in symbolic, long-horizon reasoning demands mapping that terrain through fine-grained ablations like those introduced here.