How to recognize it

User agent, as it reaches our logs:

Mozilla/5.0 (compatible; AhrefsBot/7.0; +http://ahrefs.com/robot/)

Match on the token AhrefsBot. Site Audit is a different bot with its own token, AhrefsSiteAudit: Mozilla/5.0 (compatible; AhrefsSiteAudit/6.1; +http://ahrefs.com/robot/site-audit).

  • robots.txt token: AhrefsBot
  • IP lists (JSON): crawler-ip-ranges (CIDR ranges) and crawler-ips (single addresses)
  • Reverse DNS: the hostname always ends in ahrefs.com or ahrefs.net.

What it does

AhrefsBot crawls the web for the Ahrefs index: pages, links and content that Ahrefs customers see in its tools. The same crawl powers Yep, Ahrefs’ search engine.

Site Audit runs as a separate crawler, AhrefsSiteAudit, which a customer points at a site. It obeys robots.txt by default, but verified owners can let it ignore robots.txt on their own sites. Ahrefs limits it to 30 URLs per minute.

What you gain by allowing it

  • An accurate picture of your site in Ahrefs: your pages, internal links and outgoing links. Anyone vetting your site with Ahrefs (partners, agencies, link builders) sees current data.
  • Credit for your outgoing links. Sites you link to see those links in their Ahrefs reports.
  • A presence in Yep’s results, a small source of search traffic.

What you lose

By allowing it

  • Crawl load. Ahrefs honours Crawl-delay, except when it renders JavaScript.
  • Competitors can study your pages and outgoing links in Ahrefs.

By blocking it

  • It does not hide your backlinks. Ahrefs finds links to your site by crawling the sites that link to you, not yours. Its organic keyword data comes from search results, not from crawling you either.
  • What it does hide: your own pages, internal structure and outgoing links, plus your presence in Yep. This is why private link networks block AhrefsBot: the point is to hide the links they give, not the ones they get.

How to verify it

Forward-confirmed reverse DNS: the hostname must end in ahrefs.com or ahrefs.net, and resolving it must return the original IP.

host <ip>
host <hostname from the first command>

Or match the IP against crawler-ip-ranges. The file has the same shape as OpenAI’s lists, so the one-liner on the GPTBot card works with its URL.

Cloudflare marks the real AhrefsBot as a verified bot in the category “Search Engine Optimization”. Our trackers do not verify it themselves yet, so Cloudflare’s flag is the only verdict we store.

Gotchas and history

  • One rule for SEO crawlers often catches AhrefsSiteAudit too. You need that bot if you run Ahrefs Site Audit on your own site, so either exempt it by name in the rule, or keep AhrefsBot out with robots.txt instead of the firewall.
  • Do not allowlist by network. The published ranges include blocks at OVH, a hosting provider anyone can rent from, so an ASN rule for “Ahrefs” would admit strangers. Use the IP list or reverse DNS.
  • Ahrefs gives no time for robots.txt changes, only that they are picked up before the next scheduled crawl.
  • Ahrefs’ robot page does not say whether crawled content is used to train AI models.

What we observed

Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).

  • 123 requests, all with the documented AhrefsBot/7.0 user agent.
    • Cloudflare flagged 120 as a verified SEO crawler, all from AS16276 (OVH), which matches the OVH blocks in Ahrefs’ published ranges.
    • The other 3 came from a different network, without the flag.
  • A firewall rule on both sites answered every request with 403 at the edge, including 41 requests for /robots.txt. The bot never got to read our robots.txt and kept coming back on all 8 days. A firewall block tells a polite crawler nothing; a Disallow it can read is what Ahrefs documents.
  • AhrefsSiteAudit (38 requests in the same period) hit the same rule.

About Ahrefs

  • AhrefsBot crawls the whole web for the Ahrefs index and the Yep search engine. It obeys robots.txt and Crawl-delay.
  • AhrefsSiteAudit crawls one site at a time for a customer’s Site Audit project, at most 30 URLs per minute. It obeys robots.txt by default, but a verified owner can make it ignore robots.txt on their own site.

The tokens are separate. Blocking AhrefsBot to keep competitors out does not have to break your own Site Audit, but a firewall rule matching “Ahrefs” in the user agent blocks both.

Sources