Claude Limitations in 2026: A Documented, Sourced List

Claude’s own system cards admit more than most vendors do — Sonnet 5 doesn’t come close to Opus-class cybersecurity or research-automation ability, its 1M-token context window loses roughly a quarter of multi-needle retrievals in controlled tests, and Anthropic has changed Claude Code’s usage caps four times since March 2026, most recently letting a 50% weekly-limit bonus expire on July 13. This page is a documented, sourced ledger of Claude’s limitations as they stand in September 2026. Every claim below carries a model version, a source, and a date. Where trackers disagree, the disagreement is reported, not smoothed over — that’s the same standard we hold ChatGPT to in our companion ledger entry, and the same methodology laid out at the Limitations Ledger hub.

Scope and method

“Claude” in late 2026 spans four active tiers: Opus 5 (released July 24, 2026, knowledge cutoff May 2026, 1M-token context, $5/$25 per million input/output tokens), Opus 4.8 (released May 28, 2026), Sonnet 5 (released June 30, 2026, Anthropic’s new default mid-tier), and Haiku 4.5. A limitation measured on one tier does not automatically transfer to another — Sonnet 5 trails Opus 4.8 on coding and cybersecurity evaluations by Anthropic’s own account, while Haiku 4.5 posts the lowest hallucination rate of the three on constrained tasks. Each entry below names the specific model and evaluation, following the same rule we use when evaluating LLMs generally: no score without a named benchmark version and date.

Hallucination: the rate depends entirely on what you measure

Claude’s hallucination numbers vary by roughly an order of magnitude depending on the task family, which is itself the finding worth reporting. On BenchLM’s aggregation, Claude Opus 4.8 posts a 35.9% hallucination rate against an AA-Omniscience accuracy index of 27 — second only to Gemini 3.1 Pro‘s 33 — while Claude Sonnet 5 comes in at 38.3% accuracy and 37.3% hallucination. Those are open-ended, abstention-testing numbers: the model is penalized for guessing instead of declining to answer.

A different, narrower methodology tells a more flattering story. A five-model Digital Applied study (2026) testing constrained, document-grounded summarization put Claude Haiku 4.5 at the low end of the entire cohort, 4.62%, with GPT-5.4-mini the high end at 6.10%. Both figures are “Claude’s hallucination rate.” They differ by a factor of eight because one measures open-domain guessing discipline and the other measures grounded-summary faithfulness — the same trap we’ve flagged in benchmark saturation generally. A further wrinkle specific to Claude and its reasoning-mode peers: models run in extended-thinking mode have been shown to hallucinate at roughly 2–3x the rate of their non-reasoning counterparts on some trackers, which cuts against the common assumption that “thinking harder” reduces factual error. For a full comparison of grounded, closed-book, and calibration hallucination benchmarks across every frontier model, see how hallucination is actually measured.

Opus 5, released July 24, 2026, sharpens this trade-off rather than resolving it. On AA-Omniscience, Artificial Analysis measured Opus 5 gaining 7 points of accuracy over Opus 4.8 while its hallucination rate rose 14 points to 50% — AA’s own framing is that the model “answers more often when uncertain” (Artificial Analysis, July 24, 2026). BenchLM’s independent profile reports the same 50% figure, attributing it to the identical mechanism. Two trackers agreeing on both the number and the cause is about as solid as this kind of claim gets — and it means the newest, most capable Claude is also the one that bluffs most. AA notes Opus 5 still trails Claude Fable 5 on factual knowledge, which it attributes to size class rather than training choices.

Update, September 28, 2026: Anthropic released Opus 5.5 on September 22, 2026 — the first model in a new 5.5 family, priced at $4/$20 per million input/output tokens (a 20% cut from Opus 5’s $5/$25) with cache reads down 60% to $0.20/million. Anthropic’s own comparison table shows Opus 5.5 scoring 66.4% on Terminal-Bench 4.0 against 52.3% for Opus 5, 55.8% for Claude Fable 5.1, and 57.9% for GPT-6 Astra, and 87.4% on the MASK truthfulness-under-pressure benchmark. Anthropic frames the release as an efficiency and cost consolidation rather than a capability leap — METR’s pre-deployment note on Opus 5.5 explicitly calls it “an incremental improvement… rather than a discontinuous jump” over Fable 5.1, and no updated time-horizon figure has been published for it yet, so the 11h59m/1h10m numbers above remain the most recent hard data on agentic reliability. Sonnet 5.5 and Haiku 5.5 are expected “in the coming weeks” per Anthropic, meaning most of Claude’s deployed volume is still running on the 5-series models this page documents (Anthropic, September 22, 2026; METR, September 22, 2026).

Update, October 7, 2026: Sonnet 5.5 is no longer forthcoming — Anthropic released it that same day, September 28, 2026, alongside Opus 5.5. It is priced identically to Sonnet 5 ($2/$10 per million input/output tokens, $0.20/million cache reads) but runs 30%+ faster and needs far fewer tokens per task, which Anthropic says cuts typical cost per task by up to 30%. On Terminal-Bench 4.0 it scores 70.6%, up from Sonnet 5’s 10.3%. More relevant to the cybersecurity finding below: Anthropic says Sonnet 5.5’s cyber capabilities are “comparable to Opus 5’s,” and it is the first Sonnet model to ship with the real-time cyber safeguards and fallbacks previously reserved for Opus-tier models — a real narrowing of the gap the Sonnet 5 system card described in June, though its biology safeguards are unchanged from Sonnet 5’s. On hallucination, Artificial Analysis’s AA-Omniscience puts Sonnet 5.5 at 47% (54% accuracy), a figure BenchLM’s independent tracker reproduces from the same underlying AA data — higher in absolute terms than Sonnet 5’s 37.3%, but the same accuracy-for-hallucination trade-off this page already documents for Opus 5 and Opus 5.5, now showing up a tier down. Haiku 5.5 has still not shipped as of this update (Anthropic, September 28, 2026; BenchLM, October 6, 2026).

Context window: 1M tokens advertised, meaningfully less delivered

Anthropic’s 1M-token context window (Opus 4.6 and later) is real as an input limit, not as a reliable-recall limit. On the MRCR v2 eight-needle retrieval test, Opus 4.6 scores 78.3% at 1M tokens — a 14% drop from its 256K-token score — meaning roughly one in four multi-needle retrievals fails even in a controlled synthetic benchmark at full context length. Independent researchers have also documented a “lost in the middle” effect specific to Claude: facts placed between roughly 30% and 70% of the way through a long context are retrieved meaningfully less reliably than facts near either end.

In production use the picture is worse than the synthetic benchmark suggests. A GitHub bug report against Claude Code documents performance degrading well before the window is half full: circular reasoning appearing at roughly 20% context usage, and automatic context compression wiping usable history at around 40% usage during heavy sessions. That’s a product-level failure mode layered on top of the model-level retrieval decay — the two compound rather than cancel.

Coding: the Verified-to-Pro gap is the tell

On SWE-bench Verified, Claude Sonnet 5 scores 85.2% and Opus 4.8 scores 88.6% — a real but modest 3.4-point gap. On SWE-bench Pro, the harder, contamination-resistant variant we cover in our SWE-bench explainer, both scores drop sharply and the gap widens: Sonnet 5 falls to 63.2% and Opus 4.8 to 69.2%. A roughly 20-to-25-point collapse between Verified and Pro for the same models on the same underlying skill is strong evidence that Verified’s headline numbers are inflated by task familiarity, not just an artifact of difficulty scaling.

Opus 5 moves both numbers up without closing the gap. On SWE-bench Verified it scores 96% per Anthropic’s own system card; on SWE-bench Pro it drops to 79.2%, behind Claude Fable 5.1’s 81.2% (BenchLM, data as of September 21, 2026). That is still a ~17-point Verified-to-Pro collapse — narrower than Opus 4.8’s ~19 points, but the same shape, eight months and two model generations later. A tracker disagreement worth naming while reading these: Vals AI’s independent SWE-bench run puts Opus 5 at 97.0% against the system card’s 96%, and on Terminal-Bench 2.1 Artificial Analysis reports 89% at max effort while Vals AI’s run of the same benchmark version returns 84.6% — a 4.4-point spread from harness and effort settings alone, on a benchmark where GPT-6 Astra leads at 87.3%. Neither is wrong; they are different scaffolds, which is exactly why this ledger refuses to quote a score without naming who ran it.

Agentic reliability, cybersecurity, and research automation

METR’s time-horizon tracking put Claude Opus 4.6 at roughly an 11-hour-59-minute task length completable at 50% reliability, but only about 1 hour 10 minutes at 80% reliability — a wide gap between “usually works” and “reliably works” that matters more for unattended agent deployments than the headline 50% figure suggests. Anthropic’s own Sonnet 5 system card (June 30, 2026) states plainly that Sonnet 5 has “a much lower ability to perform cybersecurity tasks than current Opus models,” and separately that Opus 4.8 “does not seem close to being able to substitute for Research Scientists and Research Engineers, especially relatively senior ones.” Both are self-reported capability ceilings, not third-party criticism, and both are useful precisely because they’re dated and specific rather than marketing language. The same scaffold-dependence shows up across the broader agent benchmark landscape, where the same model’s score can swing 30+ points depending on harness — see our guide to AI agent benchmarks.

Sycophancy, over-refusal, and rate limits

On a third-party sycophancy benchmark tracked by lechmazur, Claude Sonnet 4.6 was the only model in its evaluated snapshot to show any refusal behavior at all — 24 of 995 prompts (2.4%). Anthropic’s Sonnet 5 system card reports hallucination and sycophancy both improved relative to Sonnet 4.6, but flags a slight increase in “wet blanket” responses — excessively discouraging or moralizing replies to benign requests, the inverse failure mode of sycophancy rather than its absence.

Separately, Claude Code’s usage limits have changed repeatedly since March 2026, each with limited advance notice: a March prompt-caching bug drained quota 10–20x faster than intended before it was fixed; Anthropic blocked third-party OAuth tokens on April 4; five-hour limits were doubled for Pro, Max, Team, and seat-based Enterprise plans on May 6; and a temporary 50% weekly-limit bonus added May 13 was extended past its original target date before finally expiring at 11:59pm PT on September 13, 2026. Anthropic replaced it the next day with a permanent 25% increase over the pre-May baseline — which sounds like a net gain but leaves users with roughly 17% less weekly capacity than they had during the promotional period, a distinction several Max-plan subscribers flagged publicly after hitting limits faster than expected. Whether the caps move again is itself a live variable in how usable Claude Code is week to week, not a one-time policy note (Bleeping Computer, September 2026).

The ledger at a glance

Limitation	Measured result	Model	Source, date
Open-domain hallucination	50% hallucination (+14 pts vs Opus 4.8, accuracy +7)	Opus 5	Artificial Analysis & BenchLM, Jul–Sep 2026
Grounded-summary hallucination	4.62% (cohort low)	Haiku 4.5	Digital Applied study, 2026
Long-context retrieval at 1M tokens	78.3% (MRCR v2 8-needle), ~1 in 4 fail	Opus 4.6	Anthropic-reported benchmark, 2026
Context reliability in production	Circular reasoning ~20% usage; compression wipes history ~40%	Claude Code (Opus/Sonnet)	GitHub issue #35296, 2026
SWE-bench Verified → Pro drop	96%→79.2% / 88.6%→69.2% / 85.2%→63.2%	Opus 5 / Opus 4.8 / Sonnet 5	Anthropic system card & BenchLM, Sep 21 2026
Agentic reliability horizon	50%: ~11h59m task length; 80%: ~1h10m	Opus 4.6	METR, 2026
Sycophancy refusal rate	2.4% (24/995 prompts) — only model with refusals in snapshot	Sonnet 4.6	lechmazur/sycophancy, 2026
Open-domain hallucination (Sonnet tier)	47% hallucination, 54% accuracy — up from Sonnet 5’s 37.3%	Sonnet 5.5	Artificial Analysis & BenchLM, Sep–Oct 2026
Usage-cap volatility	4 policy changes Mar–Jun 2026; 50% weekly bonus expired Jul 13	Claude Code (all tiers)	Anthropic changelog via community trackers, 2026

FAQ

Is Claude’s hallucination rate better or worse than ChatGPT’s?

Neither ledger supports a single number for either model, which is the point. Claude Haiku 4.5’s grounded-task rate (4.62%) undercuts GPT-5.4-mini’s (6.10%) in the Digital Applied study, but on open-domain abstention testing, Opus 4.8’s 35.9% and GPT-5.5’s 86% aren’t measuring the same failure mode and shouldn’t be read as a head-to-head score. As of September 2026 the current-generation figures on that same AA-Omniscience test are 50% for Claude Opus 5 and 51% for GPT-6 Astra — close enough that the honest answer is neither.

Can I trust Claude’s 1M-token context window for a large document set?

For raw ingestion, yes — the tokens fit. For reliable multi-fact retrieval, budget for meaningfully less: Opus 4.6 fails roughly one in four multi-needle retrievals at full 1M-token length even in a clean synthetic test, and Claude Code sessions have shown degradation as early as 20% context usage in the wild.

Is Claude Code’s usage limit stable enough to plan around?

Not yet. Anthropic changed the caps four separate times between March and June 2026, and the most recent relief — a 50% weekly bonus — lapsed July 13, 2026. Treat published limits as a snapshot, not a contract.

Last updated October 7, 2026. This page is refreshed as benchmarks and scores move.