BigCodeBench tests LLMs on 1,140 multi-library Python tasks instead of toy problems – and its leaderboards disagree with each other more than most benchmarks do.

Reddit’s AI citation share ranges from under 1% to over 60% depending on the study. Here’s what the data actually shows, and why it disagrees so much.

A sourced ledger of real production incidents caused by autonomous coding agents: wiped databases, deleted files, leaked credentials, and runaway bills.

NIAH is saturated. RULER, MRCR, BABILong, and Fiction.liveBench show models use a fraction of their advertised context window.

GEO doesn’t replace SEO — it depends on it. Here’s what the 2026 data actually shows about ranking for clicks versus getting cited by AI.

LMArena’s Elo-style rankings come from millions of blind human votes, not a test set — here’s how the Bradley-Terry math works and where it has been gamed.

Epoch AI rated Vending-Bench 2 ‘Flawed’ on Oct 21, 2026, citing exploitable environment bugs and unequal context across providers. Here’s the leaderboard and the review.

A 2026 re-run of the original slopsquatting research found 127 hallucinated package names that five unrelated frontier models — Claude,

A federal appeals court, not a benchmark, produced the most consequential documented finding in agentic commerce so far: in August

Anthropic’s own account of its 5.5-generation models contains a detail that undercuts the evaluations built around it: Opus 5.5 reportedly