Anthropic publishes its crawler IPs. I missed it for four months
Table of contents
For four months this site said Anthropic does not publish the IP addresses its crawlers use. It does. The list lives at claude.com/crawling/bots.json, and Anthropic’s Help Center has linked to it since April 2026.
This week I added verification against that list to the tracker and put correction notes on the three posts that repeated the wrong claim. On real traffic the list holds up well, with one gap that will bite anyone who uses it as a strict allowlist: Claude-User checks robots.txt from addresses the list has never contained.
The list has been there since April
Until at least 23 March 2026, Anthropic’s Help Center article on crawling said: “we do not currently publish IP ranges, as we use service provider public IPs.” The archived copy from 10 April replaces that sentence with a link: “If a crawler has a source IP address on this list, it indicates that the crawler is coming from Anthropic.” The article dates the edit to 7 April.
The oldest archived copy of the list is from 12 April and has four entries. Every archived version since then contains all entries of the one before it.
The current version, generated on 18 August, has 26 entries and no IPv6. The first is 216.73.216.0/22, a block registered to Anthropic under the name AWS-ANTHROPIC and announced through Amazon’s network. That is why ClaudeBot traffic in my logs carries Amazon’s network number, AS16509. The other 25 are single addresses and small blocks in Google Cloud and, since August, in Microsoft Azure and AWS.
How I missed it
Anthropic documents its network in two places. The Help Center article links to the crawler list, while a separate API documentation page lists the addresses behind API and MCP traffic, including the outbound block 160.79.104.0/21, and never mentions crawlers or the crawler list.
In May I found the API page first and checked ClaudeBot against it. Almost every ClaudeBot request failed, because the crawler does not leave from that block. Three days later I retracted the check, which was right, and drew the wrong lesson: that Anthropic publishes no crawler IPs at all. My research notes from that same day already mention bots.json, but the tracker’s code comments written that day say Anthropic publishes nothing, and the comments are what spread: to the methodology page and to three posts, Rendering mode was the wrong axis, Most AI batch crawlers never fetched our robots.txt and ClaudeBot crawled our content once.
Reverse DNS would not have saved me. Anthropic does not document a hostname scheme for its crawlers, and reverse lookups on its own outbound block return other organisations: 160.79.104.1 resolves to mail.lincolncenter.org, 160.79.104.100 to fatpipe.juilliard.edu.
What the list catches
The tracker now checks ClaudeBot, Claude-User and Claude-SearchBot against the list and refreshes it every six hours. I re-ran the check over the snapshot behind the earlier posts, 18 May to 6 August (Cloudflare-logs ingest, one row per request):
| Requests | On the list | Not on the list | Could not check | |
|---|---|---|---|---|
| ClaudeBot | 871 | 834 | 10 | 27 |
| Claude-User | 8 | 3 | 3 | 2 |
The 834 ClaudeBot requests came from 50 addresses, all inside 216.73.216.0/22. The 10 failures came from three cheap VPS addresses at Infraly, Advin and SpectraIP. The same three addresses also posed as OpenAI’s, Perplexity’s, Google’s, Apple’s, Amazon’s and Meta’s crawlers in this dataset (more on that pool). The 27 I could not check come from the same Amazon network as the verified traffic, but an earlier version of the tracker had already thrown away their IP addresses.
Since the snapshot, the live tracker has checked every new ClaudeBot request against the list. As of 25 September, none has failed.
Where the list falls short
Claude-User fetches pages on a user’s behalf, when someone asks Claude about a URL. In this data it fetched pages three times from 34.162.230.222, which is on the list. Each time, in the same second, a request for /robots.txt arrived from a different Google Cloud address in 34.34.241.x. None of those three addresses appears in any archived version of the list.
The robots.txt check is good behaviour, and a strict allowlist built from bots.json flags it as fake. Block on that basis and you stop Anthropic from reading the rules you want it to follow, which is the failure the Help Center points to when it discourages IP blocking.
Claude Code is outside the list by design. Its web fetches leave from the user’s own machine, as Quercle noted in January 2025, yet its user agent starts with Claude-User, for example Claude-User (claude-code/2.1.142; ...). A classifier that matches Claude-User first lumps the two together. Mine did until May.
The list covers Claude-SearchBot as well, but that bot never visited, so I have nothing to check it against.
What this means for site owners
- Verify Anthropic’s crawlers against bots.json. Not against the API page, and not with reverse DNS. Fetch the list on a schedule: it went from 4 entries in April to 26 in August.
- Treat “on the list” as yes and “not on the list” as probably not. The Help Center only promises the first direction. For Claude-User’s robots.txt checks from Google Cloud, “not on the list” is demonstrably wrong.
- If you block by IP, block the list’s entries, not AWS.
216.73.216.0/22is Anthropic’s; blocking all of Amazon’s network (AS16509) would also block every other service that runs on it. - Classify Claude Code separately. A Claude Code request means a person pointed Claude Code at your page from their own computer; Anthropic’s crawlers were not involved.
What this does not prove
- One cold-start test domain. The addresses Anthropic uses for a large site may differ.
- The Claude-User gap rests on three requests in May. The list has changed three times since then without adding those addresses, but Claude-User has not visited again, so I cannot tell whether its robots.txt check still comes from there.
- I have not asked Anthropic whether the list is meant to be complete. The Help Center wording only commits to “on the list means Anthropic”.
Method notes
Verification runs in the tracker: ipRanges.js fetches the list every six hours and checks CIDR membership. The re-check of the snapshot used the same code on a copy of the database (backfill_verification.mjs). Counts come from the canonical Cloudflare-logs ingest, deduplicated by ray_id; the Next.js middleware ingest is excluded to avoid double counting. The history of the list comes from Internet Archive captures of claude.com/crawling/bots.json between 12 April and 9 September 2026, compared entry by entry and dated by each version’s creationTime. The Help Center text comes from the 23 March and 10 April 2026 captures. Registry ownership is from ARIN RDAP and reverse DNS from live lookups, both on 25 September 2026.
Data availability: JS SEO Lab publishes methodology, tracker code, and classifier in the public repository at github.com/Qbeczek1/jsseo-dev. Live dashboard at /dashboard/.
Bias disclosure: I run JS SEO Lab as an independent technical SEO research project. I also do paid technical SEO and AI-visibility audits through FratreSEO. No framework vendor, crawler vendor, search engine, or AI company funds this work.