Open methodology · Open data
Empirical research on what JavaScript content patterns deliver to bots and LLM fetchers in 2026. Open methodology, open data, reproducible findings.
Eight JavaScript content patterns × five page types × three rendering modes, open to every crawler. Per-cell signals captured in a public tracker.
Live test bed: next.jsseo.dev: 55 cells running now.
Render probe: one page on this site that records which crawlers run JavaScript here.
Raw bot tracker data, hypothesis pre-registration, methodology - all open from day one. CC0 / CC-BY for data, MIT for code.
Senior technical SEO consultant, 25 years in the field. Independent owner of SAWA PROJEKT, co-founder of FratreSEO.
About · jakubsawa.pl · LinkedIn · YouTube
Over three weeks, OpenAI's training crawler fetched our Next.js bundle page by page, and the React code in it never ran. Its other agents, the search crawler, the robots.txt readers and ChatGPT-User, fetched no JavaScript at all. A request for a .js file is a download, not a render. The same goes for llms.txt: the first verified OpenAI fetch of ours came a day after a page linked to it.
We disallowed 11 of 55 test pages for every bot. GPTBot, which never fetches robots.txt, stopped crawling them. Meta read the new rules and crawled them minutes later.
jsseo.dev has served an llms.txt since May. In the 30 days to 26 October, 31 outside requests asked for it, and not one came from a verified bot: no GPTBot, ClaudeBot, PerplexityBot, Googlebot or Bing...
Reflection AI's crawler visited our test bed 69 times between 18 September and 25 October. On 12 of its 13 visits to client-side-rendered pages, a second address fetched the page again and ran the Rea...
In August, Meta's AI crawler visited with a Chrome-based user agent and executed both our inline script and the React app, usually within a second of fetching the page. 83 beacons between 10 and 20 Au...
139 days of logs from one test domain. GPTBot made 805 requests and never asked for /robots.txt. In 135 of its 190 crawl sessions, a robots.txt request from an OAI-SearchBot variant that carries a rob...
Chinese AI assistants have few users in Europe and North America, yet their makers crawl Western sites, and Western companies build on their open-weight models. I found nothing published on what these...
In 2024 Vercel and MERJ reported that major AI crawlers don't execute JavaScript. That's the rendering-mode answer, and on our cells it mostly holds, with exceptions: as of 2026-11-13, ClaudeBot, Amazonbot and CCBot never fired a beacon and GPTBot only ran the inline script, but Applebot rendered React, and meta-external and ShapBot ran both the inline script and React. Real client audits don't ask "is this site SSR or CSR?" though: they ask "is this specific JS pattern (a click-to-reveal, a useEffect-fetched price, a hash-routed navigation) costing me visibility in this specific bot's index?"
Eight JS content patterns (clean / js-images / js-links / click-reveal / js-fetched / hash-routing / late-loaded / mixed) × five page types × three rendering modes (SSR / CSR / optional SSG). As of 2026-11-13, 34 named bots had visited the test cells, among them verified GPTBot, ClaudeBot, Googlebot, Bingbot, Applebot, ChatGPT-User and Claude-User.
Bot claims are checked wherever the vendor publishes a way to check them: reverse DNS for Google/Bing/Apple/Baidu, IP-range CIDR matching against published OpenAI/Perplexity/Anthropic/Amazon/Parallel manifests, and either path for Amazonbot and CCBot. Meta's crawlers are checked against Meta's own network (AS32934). Claude Code fetches stay visible but unresolved, since there is no public verification path. Deprecated anthropic-ai/Claude-Web UAs and the Google-Extended/Applebot-Extended tokens, which their vendors never send as a user agent, are automatically flagged as unverified. As of 2026-11-13, 47,838 of 57,631 hits (83%) have no verification result; 43,740 of them come from human visitors and clients the classifier can't name.