How to recognize it

User agents, as they reach our logs:

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_1) AppleWebKit/601.2.4 (KHTML, like Gecko) Version/9.0.1 Safari/601.2.4 facebookexternalhit/1.1 Facebot Twitterbot/1.0
  • First string: Meta’s crawler.
  • Second string: not Meta. It is the user agent that Apple’s Messages app is widely reported to send when it builds a link preview on the user’s own device. It names Facebook’s and Twitter’s bots so that sites hand it their preview tags.

Match on facebookexternalhit, then check the network.

  • robots.txt token: facebookexternalhit
  • IP list: none published. We check Meta’s network, AS32934.
  • Reverse DNS: none published.

What it does

When someone shares a link in Meta’s apps, this crawler fetches the page to build the preview: title, description and thumbnail image. Meta says it may bypass robots.txt when it performs security or integrity checks.

What you gain by allowing it

  • A proper preview card when your links are shared, which is what gets them clicked.

What you lose

By allowing it

  • Repeated fetches of shared pages. On our WordPress site the homepage alone was fetched 1,230 times in 8 days.

By blocking it

  • Shared links appear without a title, description or image.

How to verify it

Check that the IP belongs to Meta’s network, AS32934. Cloudflare flags the real crawler as a verified bot in the category “Page Preview”. The Apple Messages variant comes from ordinary consumer networks, so it fails the network check without being an attack.

Gotchas and history

  • Failed checks are not always attacks: a failed check on this name usually means an iPhone or Mac building a preview, not someone spoofing Meta. On our sites the scanner never used this name.
  • robots.txt reads: unlike Meta’s AI crawlers, it reads robots.txt often (see below).

What we observed

Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).

  • 1,433 requests carried this name:
    • 1,374 (96%) came from Meta’s network, and Cloudflare flagged all of them;
    • 1,307 of the Meta requests went to the WordPress site, 1,230 of them to its homepage, and 115 were for /robots.txt.
  • 56 of the 59 others used the Apple Messages user agent, from 13 different consumer networks. The scanner sent none.

About Meta

  • meta-externalagent crawls for uses such as training Meta’s foundation AI models.
  • meta-webindexer crawls to improve Meta AI’s search results, and allowing it helps Meta AI cite and link your content.
  • meta-externalfetcher fetches individual links at a user’s request, and may bypass robots.txt.
  • facebookexternalhit builds link previews when a page is shared, and may bypass robots.txt for security or integrity checks.
  • meta-externalads crawls for Meta’s advertising and business products. We have not seen it in our logs.

The training and search crawlers have separate tokens. To be cited by Meta AI without feeding training, allow meta-webindexer and disallow meta-externalagent.

Meta publishes no IP list or reverse DNS scheme on its crawler page. It calls IP allowlisting the more secure option, but refers site owners to [email protected] for the addresses. A check against Meta’s network, AS32934, proves a request came from Meta, but not which of these agents sent it.

Sources