Bravebot
Brave Search's crawler. Brave says it has no separate user agent and follows Googlebot's permissions, but it announces itself as Bravebot.
| Operator | Brave |
|---|---|
| Type | AI: search index |
| robots.txt token | None documented |
| Follows robots.txt | Partly; Crawl-delay not documented |
| Sends visitors | Yes |
| Links to your pages | Yes |
| Can end up in AI training | Not documented |
| Verification |
None published
Our tracker does not verify it yet. |
| Our call |
Publisher: Allow E-commerce: Allow Expert site: Allow What the profiles mean |
How to recognize it
User agents, as they reach our logs:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Bravebot/1.0; +https://search.brave.com/help/brave-search-crawler) Chrome/W.X.Y.Z Safari/537.36
Mozilla/5.0 (compatible; Bravebot/1.0; +https://brave.com/search/)
- First string: the real one, flagged by Cloudflare. It sends
Chrome/W.X.Y.Zliterally, a template nobody filled in, and links to Brave’s crawler help page. - Second string: what the Google Cloud scanner sent. It is a plausible fake with a different link.
Match on Bravebot, then tell them apart by the link and the network.
- robots.txt token: none documented. Brave’s crawler follows your Googlebot rules.
- IP list: none published. The real requests came from Hetzner addresses.
- Reverse DNS: none published.
What it does
Brave Search runs its own index, and this crawler fills it. Brave says it will not crawl a domain or page that Googlebot cannot crawl. It also says robots.txt is not the way to keep a page out of its index; use noindex for that. Brave’s help page says the crawler does not advertise a differentiated user agent, which the traffic on our sites contradicts.
What you gain by allowing it
- Appearing in Brave Search, and in the products that build on Brave’s index, with links.
What you lose
By allowing it
- Crawl load. On our sites half of its verified requests ended in redirects.
By blocking it
- Brave Search. You cannot block it in robots.txt without blocking Googlebot, because Brave follows your Googlebot rules. A firewall rule on the user agent is the only separate switch.
How to verify it
Brave publishes no IP list or reverse DNS pattern. Cloudflare flags the real crawler as a verified bot in the category “AI Search”. That flag, the help-page link in the user agent, and a non-Google-Cloud network together separate it from the scanner’s copies.
Gotchas and history
- Tied to Googlebot: a
Disallowaimed at Googlebot also stops Brave, and there is no way in robots.txt to treat them differently. - Out of date docs: the help page denies a separate user agent, yet the crawler identifies itself and links to that same page.
- Template:
Chrome/W.X.Y.Zin a user agent usually suggests a sloppy scraper. Here it is the real thing.
What we observed
Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).
- 1,696 requests carried a Bravebot user agent:
- 935 were flagged by Cloudflare, all with the first string, from Hetzner networks (AS212317, AS24940);
- 748 came from the Google Cloud scanner with the second string.
- The real requests covered 688 paths on all four hostnames, almost all on the static site:
- 480 got a redirect;
- 40 got an error from the origin server;
- 13 asked for
/robots.txt.
About Brave
Brave Search runs one crawler. Brave’s help page says it has no separate user agent and does not crawl what Googlebot may not crawl, so a site’s Googlebot rules decide. In practice it announces itself as Bravebot/1.0. Brave publishes no IP list or reverse DNS pattern, and uses noindex, not robots.txt, to drop pages from its index.