Answer Engine Optimisation observations: what content survives in AI-generated responses (ChatGPT, Claude, Gemini, Perplexity) and what doesn't.
Across 871 requests, ClaudeBot spent 80% on /robots.txt and /sitemap.xml and 18% on content. The shape explains it: it swept the test bed once through 2026-07-05, then switched to a machine-regular he...
Our JS-execution beacon fires through two channels - an inline script and a React client component. Over 80 days the bots sort into four tiers: renderers that evaluate React but skip inline scripts (A...
We flagged most of our Perplexity and ChatGPT crawler traffic as spoofed. Then we checked whether that was our mistake. It was not. The entire apparent Perplexity, ChatGPT-User and OpenAI-search 'craw...
In 15 days of JS SEO Lab data, the Next.js test bed saw 6,125 hits across 25 user-agent classes. GPTBot, Meta-external, Amazonbot, and three Google sub-bots together produced 862 hits with zero reques...
We thought NotebookLM was scraping static HTML. A runtime-entropy probe proved otherwise: NotebookLM executed React, generated a UUID at runtime, fetched client-side content, and only appeared once we...
In the first week of JS SEO Lab data, OAI-SearchBot made 10 requests. Every one hit robots.txt. No homepage, no sitemap, no content cell. That does not prove how OpenAI crawls every site, but it does ...