AI Crawlers Do Not Render JavaScript

There is a technical assumption running through most marketing and engineering teams right now that is quietly costing brands their AI search visibility.

The assumption is this: if Google can crawl and render our site, AI crawlers can too.

It is wrong. And the gap between what Google can do and what AI crawlers actually do is where a significant portion of AI citation invisibility originates.

GPTBot, ClaudeBot, and PerplexityBot the crawlers that feed ChatGPT, Claude, and Perplexity respectively generally fetch raw HTML rather than execute client-side JavaScript in most tested scenarios. That means if your most important content only appears after JavaScript runs, AI crawlers are likely seeing an empty shell where your product descriptions, comparison tables, FAQs, and pricing details should be.

A page that ranks on Google and converts visitors can be completely invisible to the AI engines that are now driving 87% of AI referral traffic. Those two facts coexist silently which is why this problem compounds before most teams notice it.

What AI Crawlers Actually See

When GPTBot, ClaudeBot, or PerplexityBot fetches a page, what they receive in most tested scenarios is the raw HTML response before any JavaScript executes. For a traditional server-rendered site, that raw HTML contains the full page content headings, body copy, structured data, internal links, schema markup. Everything an AI engine needs to understand, trust, and cite the page.

For a modern single-page application or a heavily client-rendered site, the raw HTML response frequently contains something very different:

• A page shell with an empty <div id="app"></div>
• Script bundle references
• Minimal metadata
• Boilerplate navigation
• No meaningful body content

The actual content the product descriptions, feature comparisons, documentation, pricing context, FAQ answers exists only in the rendered DOM after JavaScript executes. AI crawlers that do not execute JavaScript never see it.

Research from Vercel and independent testing by technical SEO practitioners including Glenn Gabe has consistently reinforced this pattern: when critical content is injected client-side, AI crawlers frequently miss it entirely. The practical conclusion from observed behavior is clear enough to act on major AI crawlers generally do not render client-side JavaScript in most tested scenarios.

One important nuance worth preserving: there are meaningful differences between training crawlers, retrieval crawlers, and the user fetch systems AI engines use at answer-generation time. These systems do not necessarily behave identically, and public documentation on each is limited. The conservative and correct approach is to assume rendering limitations across all of them unless you have tested otherwise.

Why This Is Different From the Google Problem

Technical SEO teams that have solved JavaScript rendering for Google cannot assume the problem is solved for AI crawlers. The two situations are not equivalent.

Google has invested years and substantial infrastructure into rendering JavaScript at scale. Googlebot has a rendering pipeline specifically designed to execute JavaScript and index the resulting DOM. That infrastructure is mature, well-documented, and the subject of extensive testing by the SEO community.

AI crawlers do not have equivalent rendering infrastructure in most observed cases. They are optimized for a different task retrieving content to inform answer generation and their crawling behavior reflects that different purpose. The result is a gap between what Googlebot sees and what GPTBot sees that can be significant for sites with client-rendered content.

This also means that the JavaScript SEO work your team did for Google does not transfer automatically to AI search. A site that resolved its Google rendering issues by implementing dynamic rendering specifically for Googlebot may still be serving AI crawlers an empty HTML shell if those crawlers are not included in the rendering logic.

The Pages Most at Risk

Not all pages carry equal risk. The pages where JavaScript rendering limitations cause the most AI search damage are the ones with the highest citation value the exact pages you most need AI engines to be able to read.

Product and feature pages. ChatGPT cites product pages at 20.1% of all citations according to Lantern's February 2026 data. If your product pages load their core descriptions, specifications, and pricing via client-side JavaScript, the engine most likely to cite them directly cannot read them.

Comparison and alternative pages. Comparison content earns 9.3% of all AI citations and carries disproportionate purchase-intent value. A comparison table rendered client-side is invisible to AI crawlers meaning the content most likely to influence a buyer in active evaluation is the content most at risk of not being cited.

Documentation and FAQ pages. AI engines heavily favor structured Q&A content for information and evaluation queries. FAQ pages and documentation that render their content via JavaScript are underperforming their citation potential regardless of how well they are written.

Category and landing pages. These pages establish topical authority signals that AI engines use to evaluate a brand's expertise in a given area. If the substantive content on these pages is client-rendered, that authority signal is weakened or absent.

How to Test Whether AI Crawlers Can Read Your Site

Do not assume. Test directly using methods that approximate what AI crawlers actually receive.

View Source test. Open any critical page in your browser and select View Source not Inspect Element, which shows the rendered DOM, but View Source, which shows the raw HTML response. This is the closest approximation of what an AI crawler receives. Scan the raw HTML for your primary heading, body copy, product descriptions, FAQ content, and internal links. If they are absent from View Source but visible in the rendered page, you have a rendering gap.

JavaScript-disabled test. Load the page with JavaScript disabled in your browser settings. If the page becomes an empty shell or loses its substantive content, AI crawlers are likely seeing the same empty version.

cURL test. Run a cURL request against the page URL from the command line. The response is the raw HTML exactly what most bots receive on their first fetch. Check it for content completeness the same way you would check View Source.

Server log analysis. Review your server logs for visits from GPTBot, ClaudeBot, PerplexityBot, and other documented AI crawler user agents. Check which URLs they are hitting, how frequently, and what status codes they receive. A pattern of AI crawler visits to pages with thin raw HTML is a direct signal of rendering-related visibility loss.

Controlled comparison test. Create two equivalent pages one with content delivered in HTML, one with the same content injected via JavaScript. Monitor over four to six weeks whether AI engines cite or surface one version more consistently. This is the most direct test of the citation impact of rendering decisions.

The Technical Fixes

The goal is not to abandon modern frameworks. It is to ensure the content AI engines need to cite your site is delivered in a form they can reliably consume.

Server-side rendering. SSR sends meaningful HTML from the server before any JavaScript executes. For content-heavy pages documentation, editorial content, product detail pages, category pages SSR is the cleanest and most comprehensive fix. The entire page content is available in the raw HTML response, eliminating the rendering gap entirely for those pages.

Prerendering. For sites where full SSR is architecturally complex to implement, prerendering generates static HTML snapshots of pages that would otherwise rely on client rendering. These snapshots are served to bots while users continue to receive the full JavaScript-powered experience. Prerendering is particularly effective for marketing pages, evergreen resources, and framework-driven sites with limited dynamic personalization requirements.

Static site generation. For content that is stable and does not require dynamic personalization, static generation produces fast, fully crawlable HTML with minimal rendering dependencies. Blog content, documentation, and resource pages are strong candidates.

Hybrid rendering with HTML-first content. For pages that require dynamic functionality alongside static content, the correct approach is to deliver the substantive content headings, body copy, product descriptions, FAQs, pricing context in the initial HTML response, and layer dynamic enhancements on top via JavaScript. A reliable rule of thumb: if the content matters for citation, trust, or conversion, it belongs in the initial HTML.

llms.txt The Additional Layer

Beyond rendering fixes, there is a complementary implementation that directly addresses how AI crawlers understand your site structure: llms.txt.

As covered in Lantern's site audit signals post, llms.txt is an emerging standard that provides AI crawlers with a structured, plain-language description of your site's content, purpose, and how its information should be interpreted. Where rendering fixes ensure AI crawlers can read your content, llms.txt ensures they understand its context.

The two implementations work together. A site with SSR-delivered content and a well-configured llms.txt file gives AI crawlers both the raw content to extract and the organizational context to attribute it correctly. A site with only one of the two is leaving value on the table. For teams working through the technical implementation checklist, rendering fixes come first llms.txt is the complementary layer that amplifies the impact of those fixes.

What Readable HTML Should Contain

Once the rendering architecture is correct, the initial HTML response for every high-value page should contain the following at minimum:

• Primary H1 heading with a specific, descriptive claim
• Core body copy delivering the page's primary answer or value proposition
• Product or service descriptions with specific outcome language
• Pricing context where appropriate
• Key FAQ questions and answers
• Supporting facts, statistics, and claims with source attribution
• Author and publisher signals
• Internal links to related pages
• Canonical tag and metadata
• Structured data schema as a supporting layer alongside visible text

The structured data point is worth emphasizing. Schema markup is frequently misunderstood as a substitute for visible content. It is not. AI crawlers extract meaning from the visible text of a page first. Schema markup reinforces and structures that meaning it does not replace it. A page with excellent schema but thin visible content is not a well-optimized page for AI citation purposes.

Measuring the Impact

The rendering gap is invisible in most standard analytics setups. A page can appear healthy in Google Search Console ranking well, receiving impressions, generating clicks while being completely absent from AI-generated answers because its content is unreachable by AI crawlers.

The measurement layer that makes this visible is AI citation monitoring tracking which pages on your site are being cited in ChatGPT, Perplexity, Gemini, and Claude responses, and correlating citation rates with the rendering characteristics of those pages. Pages with server-rendered content that are not being cited despite strong topical authority signal a content quality or structure problem. Pages with client-rendered content that are not being cited signal a rendering problem. The distinction matters because the fixes are completely different.

Lantern's site audit surfaces rendering-related visibility gaps automatically identifying pages where JavaScript dependency is likely reducing AI crawler access and flagging them alongside the citation performance data that shows the business impact of the gap. The connection between the technical problem and the visibility outcome is visible in a single dashboard rather than requiring manual reconciliation across crawl tools, server logs, and analytics.

Key Takeaways

• GPTBot, ClaudeBot, and PerplexityBot generally fetch raw HTML rather than execute client-side JavaScript in most tested scenarios content that only appears after JavaScript runs is likely invisible to AI crawlers
• The Google rendering infrastructure does not transfer to AI crawlers JavaScript SEO work done for Googlebot does not automatically solve the AI crawler rendering problem
• The pages most at risk are the highest citation-value pages: product pages, comparison pages, FAQ and documentation pages, and category pages
• The View Source test, JavaScript-disabled test, and cURL test are the fastest ways to identify rendering gaps without specialist tooling
• SSR, prerendering, and static generation are the primary fixes the correct approach for each page depends on its content type and dynamic requirements
• llms.txt complements rendering fixes by giving AI crawlers structured context about your site the two implementations work together
• Schema markup reinforces visible content it does not replace it and should never be treated as a substitute for HTML-delivered body copy
• AI citation monitoring is the measurement layer that distinguishes rendering problems from content quality problems when citation rates underperform

Lantern's site audit identifies JavaScript rendering gaps affecting your AI search visibility and connects them directly to citation performance data. Start your free trial at asklantern.com