Claude’s own system cards admit more than most vendors do — Sonnet 5 doesn’t come close to Opus-class cybersecurity or research-automation ability, its 1M-token context window loses roughly a quarter of multi-needle retrievals in controlled tests, and Anthropic has changed Claude Code’s usage caps four times since March 2026, most recently letting a 50% weekly-limit bonus expire on July 13. This page is a documented, sourced ledger of Claude’s limitations as they stand in July 2026. Every claim below carries a model version, a source, and a date. Where trackers disagree, the disagreement is reported, not smoothed over — that’s the same standard we hold ChatGPT to in our companion ledger entry, and the same methodology laid out at the Limitations Ledger hub.
Scope and method
“Claude” in mid-2026 spans three active tiers: Opus 4.8 (released May 28, 2026), Sonnet 5 (released June 30, 2026, Anthropic’s new default mid-tier), and Haiku 4.5. A limitation measured on one tier does not automatically transfer to another — Sonnet 5 trails Opus 4.8 on coding and cybersecurity evaluations by Anthropic’s own account, while Haiku 4.5 posts the lowest hallucination rate of the three on constrained tasks. Each entry below names the specific model and evaluation, following the same rule we use when evaluating LLMs generally: no score without a named benchmark version and date.
Hallucination: the rate depends entirely on what you measure
Claude’s hallucination numbers vary by roughly an order of magnitude depending on the task family, which is itself the finding worth reporting. On BenchLM’s aggregation, Claude Opus 4.8 posts a 35.9% hallucination rate against an AA-Omniscience accuracy index of 27 — second only to Gemini 3.1 Pro’s 33 — while Claude Sonnet 5 comes in at 38.3% accuracy and 37.3% hallucination. Those are open-ended, abstention-testing numbers: the model is penalized for guessing instead of declining to answer.
A different, narrower methodology tells a more flattering story. A five-model Digital Applied study (2026) testing constrained, document-grounded summarization put Claude Haiku 4.5 at the low end of the entire cohort, 4.62%, with GPT-5.4-mini the high end at 6.10%. Both figures are “Claude’s hallucination rate.” They differ by a factor of eight because one measures open-domain guessing discipline and the other measures grounded-summary faithfulness — the same trap we’ve flagged in benchmark saturation generally. A further wrinkle specific to Claude and its reasoning-mode peers: models run in extended-thinking mode have been shown to hallucinate at roughly 2–3x the rate of their non-reasoning counterparts on some trackers, which cuts against the common assumption that “thinking harder” reduces factual error.
Context window: 1M tokens advertised, meaningfully less delivered
Anthropic’s 1M-token context window (Opus 4.6 and later) is real as an input limit, not as a reliable-recall limit. On the MRCR v2 eight-needle retrieval test, Opus 4.6 scores 78.3% at 1M tokens — a 14% drop from its 256K-token score — meaning roughly one in four multi-needle retrievals fails even in a controlled synthetic benchmark at full context length. Independent researchers have also documented a “lost in the middle” effect specific to Claude: facts placed between roughly 30% and 70% of the way through a long context are retrieved meaningfully less reliably than facts near either end.
In production use the picture is worse than the synthetic benchmark suggests. A GitHub bug report against Claude Code documents performance degrading well before the window is half full: circular reasoning appearing at roughly 20% context usage, and automatic context compression wiping usable history at around 40% usage during heavy sessions. That’s a product-level failure mode layered on top of the model-level retrieval decay — the two compound rather than cancel.
Coding: the Verified-to-Pro gap is the tell
On SWE-bench Verified, Claude Sonnet 5 scores 85.2% and Opus 4.8 scores 88.6% — a real but modest 3.4-point gap. On SWE-bench Pro, the harder, contamination-resistant variant we cover in our SWE-bench explainer, both scores drop sharply and the gap widens: Sonnet 5 falls to 63.2% and Opus 4.8 to 69.2%. A roughly 20-to-25-point collapse between Verified and Pro for the same models on the same underlying skill is strong evidence that Verified’s headline numbers are inflated by task familiarity, not just an artifact of difficulty scaling.
Agentic reliability, cybersecurity, and research automation
METR’s time-horizon tracking put Claude Opus 4.6 at roughly an 11-hour-59-minute task length completable at 50% reliability, but only about 1 hour 10 minutes at 80% reliability — a wide gap between “usually works” and “reliably works” that matters more for unattended agent deployments than the headline 50% figure suggests. Anthropic’s own Sonnet 5 system card (June 30, 2026) states plainly that Sonnet 5 has “a much lower ability to perform cybersecurity tasks than current Opus models,” and separately that Opus 4.8 “does not seem close to being able to substitute for Research Scientists and Research Engineers, especially relatively senior ones.” Both are self-reported capability ceilings, not third-party criticism, and both are useful precisely because they’re dated and specific rather than marketing language.
Sycophancy, over-refusal, and rate limits
On a third-party sycophancy benchmark tracked by lechmazur, Claude Sonnet 4.6 was the only model in its evaluated snapshot to show any refusal behavior at all — 24 of 995 prompts (2.4%). Anthropic’s Sonnet 5 system card reports hallucination and sycophancy both improved relative to Sonnet 4.6, but flags a slight increase in “wet blanket” responses — excessively discouraging or moralizing replies to benign requests, the inverse failure mode of sycophancy rather than its absence.
Separately, Claude Code’s usage limits have changed four times between March and June 2026, each with limited advance notice: a March prompt-caching bug drained quota 10–20x faster than intended before it was fixed; Anthropic blocked third-party OAuth tokens on April 4; five-hour limits were doubled for Pro, Max, Team, and seat-based Enterprise plans on May 6; and a temporary 50% weekly-limit bonus added May 13 expired on July 13, 2026 — a week before this page was published. Whether that bonus is renewed is itself a live variable in how usable Claude Code is week to week, not a one-time policy note.
The ledger at a glance
| Limitation | Measured result | Model | Source, date |
|---|---|---|---|
| Open-domain hallucination | 35.9% hallucination / AA-Omniscience index 27 | Opus 4.8 | BenchLM / Artificial Analysis, 2026 |
| Grounded-summary hallucination | 4.62% (cohort low) | Haiku 4.5 | Digital Applied study, 2026 |
| Long-context retrieval at 1M tokens | 78.3% (MRCR v2 8-needle), ~1 in 4 fail | Opus 4.6 | Anthropic-reported benchmark, 2026 |
| Context reliability in production | Circular reasoning ~20% usage; compression wipes history ~40% | Claude Code (Opus/Sonnet) | GitHub issue #35296, 2026 |
| SWE-bench Verified → Pro drop | 85.2%→63.2% / 88.6%→69.2% | Sonnet 5 / Opus 4.8 | Vals.ai / vendor trackers, 2026 |
| Agentic reliability horizon | 50%: ~11h59m task length; 80%: ~1h10m | Opus 4.6 | METR, 2026 |
| Sycophancy refusal rate | 2.4% (24/995 prompts) — only model with refusals in snapshot | Sonnet 4.6 | lechmazur/sycophancy, 2026 |
| Usage-cap volatility | 4 policy changes Mar–Jun 2026; 50% weekly bonus expired Jul 13 | Claude Code (all tiers) | Anthropic changelog via community trackers, 2026 |
FAQ
Is Claude’s hallucination rate better or worse than ChatGPT’s?
Neither ledger supports a single number for either model, which is the point. Claude Haiku 4.5’s grounded-task rate (4.62%) undercuts GPT-5.4-mini’s (6.10%) in the Digital Applied study, but on open-domain abstention testing, Opus 4.8’s 35.9% and GPT-5.5’s 86% aren’t measuring the same failure mode and shouldn’t be read as a head-to-head score.
Can I trust Claude’s 1M-token context window for a large document set?
For raw ingestion, yes — the tokens fit. For reliable multi-fact retrieval, budget for meaningfully less: Opus 4.6 fails roughly one in four multi-needle retrievals at full 1M-token length even in a clean synthetic test, and Claude Code sessions have shown degradation as early as 20% context usage in the wild.
Is Claude Code’s usage limit stable enough to plan around?
Not yet. Anthropic changed the caps four separate times between March and June 2026, and the most recent relief — a 50% weekly bonus — lapsed July 13, 2026. Treat published limits as a snapshot, not a contract.
Last updated July 21, 2026. This page is refreshed as benchmarks and scores move.