All topics

Topical index. Click a topic to see all findings related to it.

  • AEO 2026

    Answer Engine Optimisation observations: what content survives in AI-generated responses (ChatGPT, C...

  • Bing rendering

    First systematic test of Bing's JavaScript rendering behaviour since the 2019 Edge crawler note. Per...

  • Bot behaviour

    Per-bot-class observations: how Googlebot, GPTBot, ClaudeBot, ChatGPT-User and others handle differe...

  • Crawl budget

    How crawlers spend their requests on the test bed: content vs. robots.txt and sitemap fetches, crawl...

  • JavaScript rendering

    Which bots actually execute JavaScript, measured with an in-page beacon instead of inferred from fet...

  • Methodology

    Notes on how the experiment is conducted: pre-registration, verification (rDNS / IP-range / signatur...

  • JS content patterns

    The 8 JS content patterns tested: clean, js-images, js-links, click-reveal, js-fetched, hash-routing...

  • Bot spoofing

    Traffic that claims to be a known crawler but is not: how we check user agents against published IP ...