The most consistent finding across 2026 evaluations of ChatGPT is not what it can’t do — it’s what it does confidently wrong: the flagship model still answers questions it shouldn’t, retrieves reliably from only a fraction of its advertised context window, and stores memories that quietly corrupt future answers. This page is a documented, sourced ledger of ChatGPT’s limitations as they stand in July 2026. Every claim below carries a version, a source, and a date. Where trackers disagree — and they do — the disagreement is reported, not smoothed over. For the methodology behind entries like this one, see our Limitations Ledger hub.
Scope and method
“ChatGPT” in 2026 is not one model. The consumer product routes across GPT-5.x variants (GPT-5.4 shipped March 2026, GPT-5.5 in April, GPT-5.6 in early July), and a limitation measured on one variant does not automatically hold for another. Each entry below names the specific model and evaluation. We exclude anything sourced only to vendor marketing or unversioned blog claims — the same rule we apply when evaluating LLMs generally.
Hallucination: better, but the headline numbers hide a split
OpenAI’s GPT-5.5 system card (April 23, 2026) reports that individual claims are 23% more likely to be factually correct than GPT-5.4’s, and that responses contain a factual error 3% less often. That is claim-level progress — but it is not the “60% hallucination reduction” that circulated in press coverage. An independent Wire analysis (2026) traced the 60% figure to a context-engineering evaluation setup, not the model’s base factuality.
The trackers diverge sharply depending on what they measure. On Artificial Analysis’s AA-Omniscience (2026), GPT-5.5 (xhigh reasoning) posts the highest accuracy of any frontier model at 57% — while simultaneously recording an 86% hallucination rate, meaning that when it doesn’t know the answer, it guesses rather than abstains the overwhelming majority of the time. On Suprmind’s June 2026 aggregation, GPT-5.5 Pro’s hallucination rate drops from 8.3% to 4.2% with extended thinking enabled. Both numbers are “the hallucination rate,” and they differ by an order of magnitude because one measures abstention discipline and the other measures error frequency in grounded tasks. Citation accuracy remains the worst task family in that aggregation, averaging 12.4% hallucination even with extended thinking.
The historical baseline matters too: the original GPT-5 (gpt-5-main, 2025 system card) scored roughly 47% hallucination on SimpleQA at ~46% accuracy, and the SimpleQA Verified follow-up gave GPT-5 an F1 of 52.3, second to Gemini 2.5 Pro’s 55.6. Progress since then is real. Solved, it is not — a distinction that matters when hallucinated facts concern real entities, a failure mode we cover from the brand side in How AI Assistants Decide Which Brands to Recommend.
Long context: advertised 1M, usable far less
GPT-5.4 restored 1M-token context as a premium tier in March 2026, and GPT-5.6 advertises 1.05M with 128K output. Retrieval quality does not keep up with the headline number. On MRCR v2 (8-needle), GPT-5.4 scores 36.6% in the 512K–1M range (yage.ai, March 15, 2026) — against Claude Opus 4.6’s 76% at 1M on the same test. A separate 2026 needle-in-haystack analysis puts the effective multi-needle production window for GPT-5.5-class models in the 200–400K band, with most frontier models losing 15–30% retrieval accuracy between 4K and 128K on RULER-style evaluations. The practical rule: treat anything past ~400K tokens as best-effort, not dependable.
Memory: a feature that can poison its own answers
OpenAI disclosed that its pre-2025 memory system had a factual recall accuracy of 41.5% — wrong in the majority of memory-dependent situations (TechBuzz, 2026). The same testing documented ChatGPT storing outdated assumptions and incorrect personal details that then distorted every subsequent response — stale data locked in and treated as ground truth. Separately, 2026 research on memory-enabled assistants found they show a measurably stronger tendency toward sycophancy: shaping answers around what the stored profile suggests the user wants to hear. Memory has improved since the 41.5% disclosure, but OpenAI has not published a current recall-accuracy figure — an absence worth noting on a page about documented limits.
Coding and agentic work: routing regressions are real
On OPQA, OpenAI’s internal benchmark built from real research-engineering bottlenecks, GPT-5.5 passes 1.7% of tasks versus GPT-5.3-Codex’s 5.8% — a documented regression on hard, real-world coding inside OpenAI’s own system card (April 2026). And a peer-reviewed cohort study of package hallucination (arXiv 2605.17062, 2026) found 2026 frontier models still invent nonexistent software packages at rates between 4.62% and 6.10% — with GPT-5.4-mini at the top of that range. Smaller range than 2024, same security threat: a hallucinated package name is a supply-chain attack surface.
Product-level defects
Beyond benchmarks, a 2026 defect roundup documents nine reproducible issues across GPT-5.4, GPT-5, and GPT-4o in the ChatGPT product: spurious Arabic word insertion, “skeleton” code that omits promised implementations, sycophantic agreement, Enterprise SSO failures, memory regressions, and clickbait-style response endings. These are product bugs, not model limits — but users experience them identically.
The ledger at a glance
| Limitation | Measured result | Model | Source, date |
|---|---|---|---|
| Guessing instead of abstaining | 86% hallucination rate at 57% accuracy (AA-Omniscience) | GPT-5.5 xhigh | Artificial Analysis, 2026 |
| Grounded hallucination | 8.3% → 4.2% with extended thinking | GPT-5.5 Pro | Suprmind aggregation, Jun 2026 |
| Long-context retrieval | 36.6% on MRCR v2 8-needle at 512K–1M | GPT-5.4 | yage.ai, Mar 2026 |
| Memory recall | 41.5% factual recall (legacy system; no current figure published) | ChatGPT memory | OpenAI disclosure via TechBuzz, 2026 |
| Hard real-world coding | 1.7% pass vs 5.8% for GPT-5.3-Codex (OPQA) | GPT-5.5 | OpenAI system card, Apr 2026 |
| Package hallucination | Up to 6.10% invented packages (cohort high) | GPT-5.4-mini | arXiv 2605.17062, 2026 |
| Citation accuracy | 12.4% average hallucination with thinking enabled | Frontier cohort incl. GPT-5.5 | Suprmind, Jun 2026 |
How to read these numbers
Three cautions. First, hallucination benchmarks measure different things — abstention discipline (AA-Omniscience), grounded summarization (Vectara’s leaderboard), and parametric recall (SimpleQA) produce non-comparable percentages for the same model. Second, “with thinking” and “without thinking” are effectively different products; system cards increasingly report only the flattering configuration. Third, saturation pressure applies here as much as on capability benchmarks — as we documented for GPQA, once a metric becomes a marketing number, its diagnostic value starts decaying. Domain-specific results can look dramatically better than general ones: a PMC study found GPT-5 with thinking mode reached 1.6% hallucination on HealthBench versus GPT-4o’s 15.8% — true, sourced, and not generalizable beyond medical Q&A.
FAQ
Is ChatGPT’s hallucination problem solved in 2026?
No. Error frequency is down substantially — 23% better claim-level accuracy in GPT-5.5 per OpenAI’s system card — but abstention behavior remains poor: on AA-Omniscience, GPT-5.5 guesses rather than declines on the overwhelming majority of questions it cannot answer.
How much of ChatGPT’s 1M context window is actually usable?
Benchmarks put dependable multi-needle retrieval at roughly 200–400K tokens for GPT-5.5-class models. GPT-5.4 scored 36.6% on MRCR v2 8-needle in the 512K–1M range — the advertised window is real for input, not for reliable recall.
Should I turn ChatGPT’s memory off?
If your work depends on factual precision, there’s a documented case for it: the legacy memory system recalled facts correctly only 41.5% of the time, memory-enabled assistants test as more sycophantic, and OpenAI has not published a current recall figure to establish the feature has crossed a reliability threshold.
Last updated July 17, 2026. This page is refreshed as benchmarks and scores move.