Open methodology · Open data
Empirical research on what JavaScript content patterns deliver to bots and LLM fetchers in 2026. Open methodology, open data, reproducible findings.
Eight JavaScript content patterns × five page types × three rendering modes, open to every crawler. Per-cell signals captured in a public tracker.
Live test bed: next.jsseo.dev: 55 cells running now.
Raw bot tracker data, hypothesis pre-registration, methodology - all open from day one. CC0 / CC-BY for data, MIT for code.
Senior technical SEO consultant, 25 years in the field. Independent owner of SAWA PROJEKT, co-founder of FratreSEO.
Reflection AI's crawler visited our test bed 69 times between 18 September and 25 October. On 12 of its 13 visits to client-side-rendered pages, a second address fetched the page again and ran the React app within three seconds. On 56 visits to server-rendered and static pages, including pages whose main content needs JavaScript, it fetched the HTML once and ran nothing. Every address it fetched pages from is on Reflection's published IP list and resolves to reflection.ai.
In August, Meta's AI crawler visited with a Chrome-based user agent and executed both our inline script and the React app, usually within a second of fetching the page. 83 beacons between 10 and 20 Au...
139 days of logs from one test domain. GPTBot made 805 requests and never asked for /robots.txt. In 135 of its 190 crawl sessions, a robots.txt request from an OAI-SearchBot variant that carries a rob...
Chinese AI assistants have few users in Europe and North America, yet their makers crawl Western sites, and Western companies build on their open-weight models. I found nothing published on what these...
Since April 2026 Anthropic's Help Center has linked a crawler IP list at claude.com/crawling/bots.json. I checked ClaudeBot against the wrong Anthropic page, concluded no crawler list existed, and rep...
In Cloudflare's logs, ClaudeBot spent 80% of 871 requests on /robots.txt and /sitemap.xml and 18% on content. The shape explains it: it swept the test bed in three short bursts (18-20 May, 3-5 July, 1...
Our JS-execution beacon fires through two channels - an inline script and a React client component. Over 80 days the bots sort into four tiers: renderers that evaluate React but skip inline scripts (A...
In 2024 Vercel and MERJ reported that major AI crawlers don't execute JavaScript. That's the rendering-mode answer, and on our cells it mostly holds, with exceptions: as of 2026-10-06, ClaudeBot, Amazonbot and CCBot never fired a beacon and GPTBot only ran the inline script, but Applebot rendered React, and meta-external and ShapBot ran both the inline script and React. Real client audits don't ask "is this site SSR or CSR?" though: they ask "is this specific JS pattern (a click-to-reveal, a useEffect-fetched price, a hash-routed navigation) costing me visibility in this specific bot's index?"
Eight JS content patterns (clean / js-images / js-links / click-reveal / js-fetched / hash-routing / late-loaded / mixed) × five page types × three rendering modes (SSR / CSR / optional SSG). As of 2026-10-06, 28 named bots had visited the test cells, among them verified GPTBot, ClaudeBot, Googlebot, Bingbot, Applebot, ChatGPT-User and Claude-User.
Bot claims are checked wherever the vendor publishes a way to check them: reverse DNS for Google/Bing/Apple/Baidu, IP-range CIDR matching against published OpenAI/Perplexity/Anthropic/Amazon/Parallel manifests, and either path for Amazonbot and CCBot. Meta's crawlers are checked against Meta's own network (AS32934). Claude Code fetches stay visible but unresolved, since there is no public verification path. Deprecated anthropic-ai/Claude-Web UAs and the Google-Extended/Applebot-Extended tokens, which their vendors never send as a user agent, are automatically flagged as unverified. As of 2026-10-06, 31,881 of 39,962 hits (80%) have no verification result; 29,007 of them come from human visitors and clients the classifier can't name.