AI Crawler Permissions: robots.txt for GPTBot, ClaudeBot, PerplexityBot
GPTBot, ClaudeBot, and PerplexityBot are three bots each, not one. A sourced robots.txt reference from OpenAI, Anthropic, and Perplexity’s own docs.
GPTBot, ClaudeBot, and PerplexityBot are three bots each, not one. A sourced robots.txt reference from OpenAI, Anthropic, and Perplexity’s own docs.
Claude’s own system cards admit real limits in 2026: hallucination rates that vary 8x by methodology, 1M-token context that fails 1-in-4 retrievals, and Claude Code caps changed four times since March.
SWE-bench is four benchmarks, not one. OpenAI dropped Verified over contamination in Feb 2026 — here’s what each variant measures and where scores stand.
llms.txt adoption hit ~10% of sites, yet 97% of files got zero AI requests in May 2026. What the spec says, what the logs show, and the one real use case.
Every documented ChatGPT limitation in 2026 with version, source, and date: hallucination splits, usable context, memory recall, coding regressions.
GPQA Diamond went from 38.8% to ~94% in under three years. What the benchmark measures, where scores stand in July 2026, and why the last 6% may be flawed questions.
How ChatGPT, Gemini, and Perplexity assemble brand recommendations: retrieval pipelines, citation data from 230K prompts, and what the evidence says moves visibility.
A running, sourced ledger of documented failure modes in frontier AI tools: hallucination rates, context rot, agent reliability, and insecure code.
Find out if Google has indexed your page or site in 2026 using the site: operator, URL Inspection tool, and Search Console’s Pages report — plus fixes.
What Google’s BERT update was, how it changed search, and the direct line from BERT to MUM, helpful content, and 2026’s AI Overviews.