None of the major AI crawlers — GPTBot, ClaudeBot, PerplexityBot, or Meta-ExternalAgent — execute JavaScript, so any content that only appears after a script runs in the browser is, for practical purposes, invisible to the models that power ChatGPT, Claude, and Perplexity.
This isn’t a guess about crawler architecture. It’s a measured finding. A joint study by Vercel and MERJ tracked hundreds of millions of real crawler fetches across production sites and found zero evidence of client-side JavaScript execution by any AI crawler, even though several of those crawlers routinely download the JavaScript files themselves. The gap between “fetched” and “rendered” is where a large share of modern web content quietly disappears from AI training and retrieval.
What “JavaScript-rendered” actually means here
A server-rendered page ships its content inside the initial HTML response — a crawler that never runs a line of JavaScript still sees the full article, product description, or FAQ text. A client-rendered page (a typical single-page app built with React, Vue, or Angular without server-side rendering) ships a near-empty HTML shell plus a bundle of JavaScript; the actual content is injected into the DOM only after that script executes in a browser. Googlebot and Bingbot solved this years ago by running a real rendering engine. AI crawlers, as a class, have not.
That distinction determines whether a page is “AI-visible” independent of how good the writing is, how much authority the domain has, or how well the page is optimized for humans.
The evidence: what crawlers actually fetch
The most cited dataset here comes from Vercel and MERJ’s December 2024 analysis of crawler traffic against nextjs.org and two additional production sites on different tech stacks. The study is dated, but every 2026 follow-up analysis — from independent SEO researchers re-running the same server-log methodology — confirms the same behavior persists with GPTBot, ClaudeBot, and PerplexityBot unchanged.
| Crawler | Monthly fetches (Vercel network) | Executes JavaScript? | Fetches .js files anyway |
|---|---|---|---|
| Googlebot (Search + Gemini) | 4.5 billion | Yes, full rendering | 15.25% of fetches |
| GPTBot (OpenAI) | 569 million | No | 11.50% of fetches |
| Claude / ClaudeBot (Anthropic) | 370 million | No | 23.84% of fetches |
| AppleBot | 314 million | Yes, browser-based | — |
| PerplexityBot | 24.4 million | No | Fetches raw HTML only |
The pattern: GPTBot and ClaudeBot both download JavaScript files as text at meaningful rates — presumably as training corpus material — without ever executing them. Content that only exists after that script runs client-side never reaches the model. The one exception among AI-labeled traffic is Google’s Gemini, which piggybacks on Googlebot’s existing rendering infrastructure, and AppleBot, which uses a browser-based renderer similar to Googlebot’s.
What the crawler operators themselves say
OpenAI’s own crawler documentation describes GPTBot, OAI-SearchBot, ChatGPT-User, and OAI-AdsBot purely in terms of what they fetch and how robots.txt governs each — there is no claim of JavaScript execution anywhere in the spec, and community reports from developers monitoring their own server logs consistently show GPTBot requesting `.js` assets without any corresponding execution behavior (no subsequent XHR/fetch calls that a rendered page would trigger).
Anthropic’s documentation, updated in 2026, splits its crawling into three distinct agents — ClaudeBot (training-data collection), Claude-User (fetches triggered by a live user’s question), and Claude-SearchBot (search-result indexing) — each independently controllable via robots.txt. None of Anthropic’s public documentation describes a rendering engine, and third-party monitoring of ClaudeBot traffic is consistent with static HTML parsing only.
PerplexityBot’s behavior is the most unambiguous of the three: it fetches raw HTML, extracts what’s present in that response, and moves on — no rendering wait, no retry, no second pass after scripts would have finished executing.
None of this is solved by publishing an llms.txt file either — that spec addresses which pages a crawler is invited to read, not whether the HTML it fetches actually contains rendered content.
Where the line actually falls: SSR, streaming, and JSON-LD survive
This isn’t an argument against using JavaScript on your site. It’s specifically about where your content lives at the moment the HTML leaves your server. Content is AI-visible if it appears in the raw HTTP response — that includes server-side rendering (SSR), static site generation (SSG), incremental static regeneration (ISR), and React Server Components streamed into the initial payload. It also includes JSON-LD structured data placed in the page `<head>`, which is exactly why that markup remains a reliable channel for AI engines even on otherwise JavaScript-heavy stacks — see our breakdown of which schema markup AI engines actually parse.
What disappears is content injected via `useEffect`, client-side data fetching, hydration-only components, or anything rendered exclusively by a JavaScript framework running in a browser context the crawler never opens. Feature-level UI — view counters, live chat widgets, “related products” carousels — can stay client-rendered without visibility cost. Core content — the article body, the pricing table, the FAQ answers — cannot.
A second problem stacked on top: crawlers waste a third of their fetches
The Vercel/MERJ data also surfaced an efficiency problem worth flagging alongside the rendering gap: ChatGPT’s crawler spent 34.82% of its fetches hitting 404 pages, and Claude’s crawler hit 404s on 34.16% of its fetches, largely from requesting stale `/static/` assets. Googlebot, by comparison, spent only 8.22% of its fetches on 404s. Combined with high JavaScript-file-fetch rates that produce no usable content, a meaningful fraction of AI crawler budget on any given site is spent on requests that return nothing useful — which is a separate but related reason sites should keep sitemaps current and control crawler access explicitly through robots.txt rules for each bot, detailed in our AI crawler permissions guide.
Fixing it: server-render the content that matters
The practical fix is not new — it’s the same discipline JavaScript SEO has required since before AI crawlers existed. Ship core content in the initial HTML response via SSR, SSG, or ISR. Reserve client-side rendering for genuinely non-essential, interactive elements. If a full migration off client-side rendering isn’t feasible, dynamic rendering — detecting a bot user agent and serving a pre-rendered HTML snapshot while human visitors get the interactive version — is a documented middle path; tools like Prerender.io implement exactly this pattern, rendering the page once in a headless browser and caching the static output for crawler requests.
One more wrinkle worth noting for anyone measuring AI visibility: Vercel’s researchers also observed that asking ChatGPT or Claude for “fresh” information about a site sometimes produced no corresponding fetch in that site’s server logs at all — suggesting the model was answering from cached or training data rather than a live crawl, even while implying it had just checked. That’s a separate reliability question from rendering, but it reinforces the same conclusion: don’t assume an AI answer reflects your current page content just because the topic is current. For the broader question of how these visibility mechanics feed into whether AI assistants surface a brand at all, see our hub on how AI assistants decide which brands to recommend.
FAQ
Does Google’s AI (Gemini) have the same JavaScript problem as ChatGPT and Claude?
No. Gemini’s web crawling relies on Googlebot’s existing infrastructure, which has executed JavaScript for years via a full rendering engine. The Vercel/MERJ data explicitly separates Gemini/Search (Googlebot) from the non-rendering group precisely because it doesn’t share the limitation.
If I add JSON-LD structured data to a client-rendered page, does that fix visibility?
Only for the content actually encoded in that structured data, and only if the JSON-LD itself is present in the raw server HTML rather than injected by client-side JavaScript. It doesn’t make the rest of the client-rendered page readable — it’s a supplementary channel, not a substitute for server-rendering the primary content.
How do I check whether my own site’s critical content is AI-crawler-visible?
Fetch your page’s URL with a plain HTTP request (curl, or “view source” rather than “inspect element”) and check whether the content in question appears in that raw response. If it only shows up after the page finishes loading in a real browser, it was injected by JavaScript and a non-rendering crawler will not see it.
Last updated July 30, 2026. This page is refreshed as benchmarks and scores move.