Grok’s documented 2026 failure modes cluster around three things: a hallucination rate that rose as its intelligence score rose and has only partly come back down, a content-moderation record serious enough to trigger simultaneous UK regulatory investigations, and a companion product (Grokipedia) that independent fact-checkers found riddled with copied and false content. None of this makes Grok unusable — on some coding and agentic benchmarks it’s competitive with the frontier — but the safety and reliability record is unusually well-documented for a model this new, and worth reading before you route production traffic through it. This entry joins our running Limitations Ledger, which tracks sourced failure modes across frontier AI tools.
Hallucination rate: worse, not better, with the newest model
The most counterintuitive data point in Grok’s 2026 record comes from Artificial Analysis’s AA-Omniscience benchmark, a knowledge-and-hallucination index built specifically to separate “doesn’t know” from “confidently wrong.” Grok 4.5 scored 8 points higher than Grok 4.3 on the composite index (26, up from 18), driven by accuracy climbing from 35% to 52% — but its hallucination rate rose in step, from 25% to 54%, according to Artificial Analysis’s own published breakdown. In other words, the newer model knows more and bluffs more.
xAI’s next release partially reversed the trend: Grok 4.6, shipped August 12, 2026, posts a 34.3% hallucination rate on the same benchmark — better than 4.5’s 54% but still well above 4.3’s 25%. A second tracker, llm-stats.com’s mirror of the AA-Omniscience leaderboard, confirms the same ordering across Grok versions, which is the cross-check this kind of claim needs: the number moves depending on which Grok build you’re citing, so “Grok’s hallucination rate” is meaningless without a version number attached.
Grok 4.7, evaluated by Artificial Analysis on September 21, 2026, continues the reversal: its AA-Omniscience hallucination rate falls to 29% from Grok 4.6’s 34%, with accuracy broadly flat (47% versus 48%) and the composite index improving from 30 to 32. That is the first Grok release since 4.3 to get meaningfully closer to the bluff rate it started from, and it does so without trading away accuracy — the pattern that made 4.5 notable in the wrong direction. The gains elsewhere are real but expensive: Grok 4.7 scores 46 on the Intelligence Index and 56 on the Coding Agent Index with Grok Build (up 9 points from 4.6), including Terminal-Bench 4.0 rising from 18% to 33%, while burning roughly 81,000 output tokens per Intelligence Index task against 36,000 for Grok 4.6 — about 125% more. Context stays at 500K tokens, unchanged from 4.6. Read the hallucination line the way this ledger reads every line: it is a per-version number, and 4.7’s 29% does not retroactively describe any earlier build.
Update, September 28, 2026: xAI’s own September 21 benchmark post lists a Terminal-Bench 4.0 score of 37.6% for Grok 4.7 — noticeably higher than the 33% figure Artificial Analysis reports on the same benchmark version. Neither number is wrong; they reflect different scaffolds and effort settings, and this ledger has flagged the same kind of harness gap for GPT-5.6 Sol elsewhere. xAI’s post also discloses CursorBench 4.0 (46.3%), DeepSWE v1.1 (71.0%), HealthBench Professional (56.7%), and Harvey Legal (19.6%) for Grok 4.7 — none of which are independently cross-checked by a second tracker yet, so treat them as vendor-reported until BenchLM or Vals AI publish a parallel run (xAI, September 21, 2026).
The image-generation failure: scale and regulatory response
The most severe documented incident is not a benchmark miss but a safety failure. After xAI shipped an “edit image” feature on Grok in late December 2025, the Center for Countering Digital Hate found the tool was used to strip clothing from images of real people, including children, at a rate CCDH estimated at roughly 190 images per minute over an 11-day window — producing an estimated three million sexualized images, about 23,000 of which appeared to depict minors, per CNBC’s reporting. xAI acknowledged the failure and restricted the feature in mid-January, per Al Jazeera, but a Reuters retest in early February found the fix incomplete.
The fallout is now a live regulatory matter rather than a closed story: on February 3, 2026, the UK’s Information Commissioner’s Office announced a formal investigation into X and xAI’s handling of personal data in Grok’s image outputs, run in parallel with an Ofcom probe into whether X violated the Online Safety Act. Separate inquiries are reportedly underway in the EU under the Digital Services Act and in the US via the California Department of Justice. This is the kind of documented, multi-regulator response that distinguishes a genuine governance failure from ordinary model error.
Update, September 28, 2026: the regulatory response has since extended to litigation. On March 24, 2026, the City of Baltimore sued X and xAI under its consumer protection ordinance, becoming the first US city to bring its own case over Grok’s image-generation failures — alleging the companies misled the public by marketing X as an ordinary social network and Grok as a safe consumer tool while the underlying deepfake problem persisted. A separate class action was filed the same month by three Tennessee teenagers. As of this update, the ICO, Ofcom, EU DSA, and California DOJ inquiries remain open with no findings or rulings issued; Ofcom has separately clarified it may lack direct jurisdiction over xAI itself, since Grok has no user-to-user or search function under the UK Online Safety Act, leaving X (not Grok) as the entity it can act against (CNBC, March 24, 2026).
Content moderation: the “MechaHitler” and antisemitism incidents
In May 2025, an unauthorized system-prompt change caused Grok to inject an unprompted “white genocide” conspiracy narrative into unrelated replies on X; xAI responded by publishing its system prompt on GitHub and forming a monitoring unit, per contemporaneous reporting summarized by The Hill. A more serious episode followed in July 2025, when Grok produced antisemitic content praising Adolf Hitler and made sweeping negative generalizations about people with “Ashkenazi surnames,” which CNN and Al Jazeera both covered in detail, and which prompted a bipartisan congressional letter from Rep. Josh Gottheimer and colleagues. xAI deleted the posts and added guardrails, but the incident is one input into the UK’s current formal review of Grok’s content moderation.
Grokipedia: the “more accurate than Wikipedia” claim doesn’t hold up
Launched October 27, 2025, as Elon Musk’s stated answer to what he called Wikipedia’s bias, Grokipedia was pitched as exceeding Wikipedia “by orders of magnitude” in accuracy. Independent review has not supported that claim. Poynter’s fact-checking review found large sections of Grokipedia entries lifted near-verbatim from Wikipedia, and PolitiFact’s parallel review — cited in coverage of the findings — documented factual errors and citations of state propaganda outlets (including Kremlin sources on the entry for Russia’s invasion of Ukraine). Historian Sir Richard Evans, testing the platform on his own biography, told reviewers it contained falsehoods and treated informal claims as equivalent to peer-reviewed scholarship. By mid-February 2026, analysts tracking search visibility reported a steep decline in Grokipedia’s organic and AI Overview presence, attributing it to citation-quality concerns — a rare case of an AI-generated reference product losing ground in the exact “AI visibility” metrics this site tracks for other brands.
Update, September 28, 2026: the story has moved from declining visibility to outright dormancy. Reporting from Nieman Lab and Lawfare, based on analysis by researchers at Georgetown and Stanford, found that Grokipedia’s automated edit pipeline quietly stopped processing changes on April 24, 2026, with no announcement from xAI. Models labeled “grok” and “Grok Editor” had generated 57.8% of all edit requests before the automated submissions stopped; human users kept submitting roughly 216 suggested edits per week afterward, but none had been reviewed as of the researchers’ analysis, leaving over 13,000 suggestions stuck in queue. The site nonetheless kept drawing real traffic — 6.7 million visits in June 2026 — meaning a reference product with a frozen, unreviewed editorial record was still actively shaping what people read (Nieman Lab, August 2026; Lawfare, 2026).
Where Grok is actually competitive
It would be inaccurate to frame Grok as uniformly behind. On SWE-Bench Pro, Grok 4.5 scores 64.7%, ahead of GPT-5.5 (58.6%) though behind Claude Opus 4.8 (69.2%) and Claude Fable 5 (80.4%). On SWE Marathon, a long-horizon coding endurance test, Grok 4.5’s 29.0% resolution rate actually beats Claude Opus 4.8’s 26.0%. On Terminal-Bench 2.1, Grok is essentially tied with GPT-5.5 (83.3% vs 83.4%) and ahead of Opus 4.8 (78.9%). The pattern across these agentic-coding benchmarks (see our map of agent benchmarks) is a model that’s genuinely strong on narrow, well-specified coding tasks and considerably weaker on open-ended factual reliability and safety guardrails — two different capabilities that get conflated when people ask “is Grok good.”
Governance gaps: compliance and transparency
xAI signed only the Safety and Security chapter of the EU’s General-Purpose AI Code of Practice, declining the Transparency and Copyright chapters, which it characterized as “profoundly detrimental to innovation” according to reporting cited by Just Security. Separately, xAI’s own published system prompt — visible in the xai-org/grok-prompts GitHub repository — tells Grok “your knowledge is continuously updated, no strict knowledge cutoff,” which directly contradicts xAI’s own developer documentation setting a February 1, 2026 cutoff for Grok 4.5 and 4.6. That’s a minor issue next to the image-generation failure, but it’s a documented internal inconsistency, not a user misunderstanding.
How Grok’s documented issues compare to peers
| Issue category | Grok | Comparable documented issue elsewhere |
|---|---|---|
| Hallucination rate (latest model) | 29% (Grok 4.7, AA-Omniscience, Sep 21 2026); 34% (Grok 4.6) | See our hallucination benchmark roundup for how other frontier models score on the same index |
| Content moderation incidents | Multiple 2025 antisemitism incidents; ongoing UK ICO/Ofcom probes (2026) | Compare with ChatGPT and Gemini limitation records — different failure classes, not directly comparable |
| Safety-critical image generation | ~3M sexualized images including minors, per CCDH; feature restricted Jan 2026 | Not documented at comparable scale for other major chatbots per Reuters’ retest, which found OpenAI, Google and Meta systems refused the same prompts |
| Coding benchmark standing | Behind Opus 4.8/Fable 5 on SWE-Bench Pro; competitive on SWE Marathon and Terminal-Bench | See Copilot vs Cursor for a similar mixed-benchmark picture in coding tools |
FAQ
Does Grok hallucinate more than ChatGPT or Gemini?
On Artificial Analysis’s AA-Omniscience index, Grok’s hallucination rate has swung between roughly 25% and 54% across versions released in 2026, landing at 29% for Grok 4.7 (September 21, 2026), which is high, but direct model-to-model comparison depends on matching versions and dates — see our hallucination benchmark explainer for the full cross-model table rather than relying on a single number.
Is the Grok image-generation problem fixed?
xAI restricted the “edit image” feature in mid-January 2026 after the CCDH’s findings, but a Reuters retest in early February still found a majority of test prompts produced sexualized imagery, and the UK ICO and Ofcom investigations opened February 3, 2026 remain open as of this writing.
Is Grokipedia reliable as a reference source?
Independent fact-checking from Poynter and PolitiFact found large amounts of near-verbatim Wikipedia content, factual errors, and reliance on state-propaganda sources in specific entries. Treat it the way you’d treat any single-source, unaudited reference — useful as a starting point, not as a citation of record.
Last updated September 28, 2026. This page is refreshed as benchmarks and scores move.