Across multiple 2026 studies, reasoning models spend roughly an order of magnitude more tokens than standard models to answer the same question — and a meaningful share of that extra “thinking” measurably hurts accuracy instead of helping it.
Chain-of-thought reasoning models were sold on a simple trade: spend more compute at inference time, get a smarter answer. Through 2026, a cluster of independent papers has started measuring exactly how much extra compute that trade actually costs, and the honest answer is that past a certain budget, you’re often paying for nothing — or paying to make the answer worse.
The token gap, measured
LLMThinkBench (ACL 2026 Findings, also on arXiv as 2507.04023) ran 53 different LLMs and introduced an “Overthinking Score” that weighs accuracy against token efficiency rather than accuracy alone. Its headline figures: reasoning models used an average of roughly 6,780 tokens per answer against roughly 378 tokens for standard (non-reasoning) models on comparable tasks — an order-of-magnitude gap. The same paper found that capping the token budget to 1,024 tokens dropped accuracy from 72% to 44% for the models tested at that cap, but also that giving models progressively larger budgets beyond a moderate threshold produced diminishing accuracy returns.
A separate November 2025 paper, OckBench (Du, Kang, Han, Krishna, and Zhu), frames the same phenomenon as an “Overthinking Tax”: models solving the same problem at similar accuracy can differ by up to 5x in token length, with OckBench’s authors noting that smaller models in particular often incur higher real-world deployment cost specifically because their reasoning is more verbose and less efficient, not because they’re less accurate.
| Study | Key measured finding | Source / date |
|---|---|---|
| LLMThinkBench | ~6,780 tokens (reasoning) vs ~378 tokens (standard) per answer across 53 models; accuracy 72%→44% at a 1,024-token cap | ACL 2026 Findings / arXiv 2507.04023 |
| OckBench | Up to 5x token-length difference between models at similar accuracy — the “Overthinking Tax” | arXiv 2511.05722, submitted Nov 7, 2025 |
| “When More Thinking Hurts” | Extended reasoning sometimes causes models to abandon answers they had already gotten right; gains diminish at higher compute budgets | arXiv 2604.10739, April 2026 |
When thinking actively makes the answer worse
The most counterintuitive finding comes from a paper posted to arXiv in April 2026 under the (not subtle) title “When More Thinking Hurts: Overthinking in LLM Test-Time Compute Scaling”. Its authors document cases where a model reaches a correct answer early in its reasoning trace, then keeps reasoning past that point, second-guesses itself, and talks itself into a wrong final answer. The paper’s practical recommendation is that stopping generation at a moderate token budget — rather than letting a model reason until it naturally stops — can cut compute substantially while keeping measured accuracy roughly flat, because the marginal tokens beyond that point aren’t adding correct reasoning; they’re adding noise with a nonzero chance of flipping a right answer to a wrong one.
This connects directly to a documented production pattern covered in our AI coding-agent production failures piece: several of those incidents involved agents that kept “reasoning” past the point where a human would have stopped and asked a clarifying question, running up cost or taking destructive action instead. Overthinking isn’t just a cost problem in isolated benchmark runs — it shows up as a reliability problem in deployed agents too.
What vendors don’t put in the release notes
Reasoning-effort settings (low/medium/high/max, or named equivalents) are themselves a lever vendors and users can pull, and release benchmarks are frequently run at the highest effort setting a model supports — which maximizes the reported score but also maximizes token spend, often by a large multiple over a default setting. None of the three papers above claim a universal “right” budget; the consistent finding across them is that the accuracy curve flattens well before the token-spend curve does, so a model billed by API token count can look identical in reported accuracy while costing meaningfully more per query depending on which budget setting produced that score. A secondary write-up citing a “Pricing Reversal” study from Stanford and CMU, and specific multipliers like “8x tokens for 16x the cost,” turned up in this research but could not be confirmed against a primary paper — so those specific figures are flagged here as unverified rather than repeated as fact. For context on how claimed scores diverge from independently reproduced ones more broadly, see our benchmarks behind hallucination claims coverage.
Why this matters more for agents than for chat
In a chat interface, a user who waits an extra few seconds for a verbose answer rarely notices the token cost behind it. In an agentic deployment — a coding agent, a browser-use agent, a customer-service bot running in a loop — reasoning tokens are spent repeatedly across a multi-step task, and LLMThinkBench’s roughly 18x multiplier compounds every time the agent re-reasons about its next action. A workflow that calls a reasoning model dozens of times in an afternoon will feel the overthinking tax as a real line item on an API bill, not an abstract benchmark number. This is part of why token-efficiency benchmarks like OckBench frame their findings explicitly in deployment-cost terms rather than pure accuracy terms — accuracy alone understates what verbose reasoning actually costs in production.
There’s also a second-order effect worth flagging: if extended reasoning occasionally flips a correct early answer into an incorrect final one, as “When More Thinking Hurts” documents, then simply telling an agent to “think longer” or “double-check your work” — common prompting advice — is not a strictly safe instruction. It can reduce error in some cases and introduce it in others, and none of the research reviewed here gives a reliable way to predict in advance which outcome a given extra reasoning pass will produce for a given task.
Practical takeaways for anyone choosing a reasoning budget
Three things the current research supports with reasonable confidence: first, moderate reasoning budgets capture most of the accuracy gain available from “thinking” at all, and the marginal benefit of pushing further shrinks quickly. Second, token-length differences between models at similar accuracy are large enough (up to 5x per OckBench) that comparing models purely on accuracy, without normalizing for token spend, can make an inefficient model look equivalent to an efficient one when it isn’t. Third, maximum-effort settings used in vendor benchmark releases are not representative of what most production deployments will actually spend per query — treat a release-post accuracy number as a ceiling, not a realistic operating point, unless the vendor discloses the effort setting and per-query token count alongside it.
FAQ
Do reasoning models always produce better answers than standard models?
Not uniformly. LLMThinkBench’s cross-model data shows reasoning models using roughly 18x more tokens on average than standard models, and multiple studies find that pushing reasoning budgets higher past a moderate point yields diminishing or even negative returns on accuracy.
Can “more thinking” make a correct answer wrong?
Yes — “When More Thinking Hurts” (arXiv 2604.10739, April 2026) documents cases where models reach a correct answer early in a reasoning trace and then reason their way into abandoning it, producing a wrong final output.
Is there an ideal token budget for reasoning models?
No universal number; the research consistently finds a moderate-budget sweet spot beyond which accuracy gains flatten while token cost keeps climbing, but the exact threshold varies by model and task, and none of the cited studies specify one fixed figure that generalizes across models.
Last updated 2026-11-13. This page is refreshed as benchmarks and scores move.