How to recognize it

User agent, as it reaches our logs:

Mozilla/5.0 (compatible; DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot)

DataForSEO’s page shows the string without /1.0 and lists the version separately. Match on DataForSeoBot.

  • robots.txt token: DataForSeoBot
  • IP list: none as a file. DataForSEO prints three IPv4 subnets on its page, such as 136.243.228.176/29, plus three IPv6 ones.
  • Reverse DNS: names like crawling-gateway-136-243-228-176.dataforseo.com.

What it does

DataForSeoBot finds new links for DataForSEO’s backlink database and rechecks known ones. DataForSEO says it collects no other data about the site. DataForSEO sells this data through an API that other SEO products build on. The bot waits 5 seconds between visits by default and honours Crawl-delay.

What you gain by allowing it

  • Your links counted in the many tools that buy DataForSEO’s data.

What you lose

By allowing it

  • Crawl load, small: 315 requests in 8 days on our sites.

By blocking it

  • Presence in tools built on DataForSEO’s backlink data. Your backlinks stay visible elsewhere, because they are found on other sites.

How to verify it

Reverse DNS to crawling-gateway-*.dataforseo.com, confirmed forward, or a match against the subnets on its page. Cloudflare flags it as “Search Engine Optimization”. Our trackers do not verify it yet.

Gotchas and history

  • A second crawler: DataForSEO’s On-Page API runs a separate crawler for customers. Its default user agent is RSiteAuditor, customers can set any other, and an API option lets them replace your robots.txt with their own. A block for DataForSeoBot does not touch it.

What we observed

Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).

  • 313 requests flagged by Cloudflare, from Hetzner’s network (AS24940), where DataForSEO’s documented subnets sit. The scanner did not use this name.
  • On the static site, all 273 requests were answered normally.
  • On the WordPress site, a firewall rule blocked 33 of 40, including 10 robots.txt requests.

About DataForSEO

  • DataForSeoBot builds DataForSEO’s backlink database, sold through its API. It obeys robots.txt and Crawl-delay, and is verified by reverse DNS under dataforseo.com.
  • The On-Page API crawler is a separate crawler that customers run. Its default user agent is RSiteAuditor, but customers can change it and replace a site’s robots.txt with their own. A block for DataForSeoBot does not reach it.

Sources