AI Crawlers Don't Render JavaScript. Don't Assume Google's Does Either.

A page can rank well on Google and still be invisible to ChatGPT, Claude, and Perplexity. That's not a hypothetical. A 41-day controlled experiment published by Search Engine Land last month put real numbers on it, and the numbers are more one-sided than most people assume, on both sides of the story.

The assumption everyone's still working from

For years, the standard answer to concerns about JavaScript-heavy sites was simple: Google renders JavaScript, so it's fine. That got easier to believe in 2019, once Googlebot moved to an evergreen, continuously updated version of Chromium.

The problem is that answer was only ever about Google. The crawlers behind AI answers, GPTBot, ClaudeBot, PerplexityBot, Meta's crawler, Amazon's, are a different population entirely, and they don't behave like Googlebot at all.

What the experiment actually found

SEO consultant Vinicius Stanula ran the test properly. He built a site of roughly 2,400 pages, split its internal navigation so half the sections used hard-coded HTML links and half used links injected by JavaScript, then logged every bot request for 41 days.

The results for the non-Google crawlers were unambiguous. GPTBot and ClaudeBot crawled every page in the HTML-linked sections and found zero pages in the JavaScript-linked sections. Meta's crawler and Amazon's did the same. Bingbot managed three pages out of 293, and even that looked more like leakage from another discovery path than genuine rendering.

The Google side is where it gets more interesting, and less reassuring than the old assumption suggests. Google actually runs two separate crawlers here. GoogleOther, which doesn't build the search index, did render the JavaScript and found a meaningful share of the pages. Googlebot, the crawler that actually determines what shows up in Google Search, found only 2% of the JavaScript-linked pages after 27 days. That was on a brand new domain with no external links, and an established site may see more investment from Googlebot. But the headline finding still stands: on this evidence, "Google renders JavaScript" is the exception, and only a partial one.

Where this actually bites

This isn't about isolated widgets or the odd interactive element. It's about navigation and core content that only exists after the browser runs a script. Single-page apps built in React or Vue by default, componentised rebuilds where content loads client-side, and increasingly, output from AI website builders and vibe coding tools that default to client-side rendering unless someone deliberately configures otherwise, are all worth a specific check rather than an assumption.

Why Search Console won't tell you

It's easy to assume a healthy Search Console means you're covered. It doesn't, not for this. Search Console reports on Google indexing. It has nothing to say about what GPTBot or ClaudeBot can see, because there's no equivalent reporting tool for AI crawlers at all. A page can look completely fine in Search Console while being structurally unreachable to every AI assistant sending traffic your way.

How to check your own site

This is a five-minute audit, not a project. Disable JavaScript in your browser and load your most important pages, or compare the raw HTML source against the rendered DOM in your browser's dev tools. If your navigation, your key claims, or your calls to action only appear after the JavaScript runs, that's exactly what a non-Google AI crawler will miss.

What to actually do about it

Server-side rendering or static generation for anything that needs to be found, full stop. Client-side JavaScript is fine for genuinely interactive elements, it's a liability for navigation and content. And don't block AI crawlers like GPTBot or ClaudeBot in robots.txt if visibility in AI search is the goal, that solves a different problem and creates this one.

This is exactly why we run a digital audit before we touch strategy. Problems like this one don't show up in the metrics most businesses are already watching, which is precisely why they last so long. If you want to see what that looks like against a real account, our case studies cover a few. And if you're working through what AEO actually means in practice, our recent pieces on AEO for B2B and where search is heading are worth a look too.