MJ12bot
Majestic's distributed crawler for its backlink index. It runs on volunteers' machines, has no IP list, and reads a 403 on robots.txt as "disallow all".
| Operator | Majestic |
|---|---|
| Type | SEO tool: SEO index |
| robots.txt token | MJ12bot |
| Follows robots.txt | Yes, by the operator's account; Crawl-delay honoured |
| Sends visitors | No |
| Links to your pages | Not applicable |
| Can end up in AI training | Not documented |
| Verification |
None published
Our tracker does not verify it yet. |
| Our call |
Publisher: Allow, slowed down E-commerce: Allow, slowed down Expert site: Allow, slowed down What the profiles mean |
How to recognize it
User agents, as they reach our logs:
Mozilla/5.0 (compatible; MJ12bot/v1.4.8; http://mj12bot.com/)
Mozilla/5.0 (compatible; MJ12bot/v2.0.7; http://mj12bot.com/)
Majestic’s page names version 1.4.8, from 2017, as current, yet v2.0.7 is also live. Match on MJ12bot.
- robots.txt token:
MJ12bot - IP list: none. The crawler runs on volunteers’ computers, so its addresses cannot be listed, and Majestic asks site owners not to block it by IP.
- Reverse DNS: none. The only verification is an identification string you arrange with Majestic by email, sent in a
CRAWLER-IDENTheader or inside the user agent.
What it does
MJ12bot builds the link graph behind Majestic’s Site Explorer and its metrics, such as Trust Flow. Majestic says it does not cache page content or personal data. It honours Crawl-delay, but cuts values above 20 seconds to 20.
What you gain by allowing it
- Current Majestic data about your site. People who judge sites by Majestic metrics, such as link builders and partners, see your real link profile.
What you lose
By allowing it
- Crawl load from many machines at once, and repeat visits to URLs that answered 404 or 301 long ago. It also follows
nofollowlinks.
By blocking it
- Majestic’s view of your site, but not your backlinks: those are found on the sites that link to you.
How to verify it
There is no technical check without arranging an ident string with Majestic. Cloudflare flags the real crawler as a verified bot in the category “Search Engine Optimization”, which is the practical test. A blocklist by IP will not work against a volunteer network.
Gotchas and history
- The
*group: a group for MJ12bot replaces the*group entirely. ACrawl-delayset under*does not apply to it unless repeated in its own group. - robots.txt errors: Majestic reads robots.txt answers differently from Google:
- an unreachable file means “crawl allowed”;
- some errors, such as 403, mean “disallow everything”;
- Google reads a 403 as “no restrictions”, so the same firewall block tells the two crawlers opposite things.
- Redirects: it follows robots.txt redirects only within the same domain.
What we observed
Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).
- 831 requests from 50 addresses in 7 networks, mostly OVH and Hetzner. Cloudflare flagged 825 of them, and the scanner never used this name.
- On the static site, all 787 requests were answered normally.
- On the WordPress site a firewall rule started blocking SEO crawlers on 20 October. After that every request got a 403, robots.txt included (14 times), which by Majestic’s own rules reads as “disallow everything”.
About Majestic
Majestic runs one crawler, MJ12bot, for the link index behind Site Explorer. It is a distributed crawler on volunteers’ computers, so it has no IP list. Majestic asks site owners not to block it by IP and offers an arranged identification string instead.