Applebot-Extended
A robots.txt token, not a crawler. It decides whether content Applebot crawls may train Apple's foundation models. It does not affect Apple's search.
| Operator | Apple |
|---|---|
| Type | AI, robots.txt token only: AI training |
| robots.txt token | Applebot-Extended |
| Follows robots.txt | Yes, by the operator's account; Crawl-delay not honoured |
| Sends visitors | No |
| Links to your pages | No |
| Can end up in AI training | Yes |
| Verification |
None published
Our tracker marks every request as fake, because no real request uses this name. |
| Our call |
Publisher: Block E-commerce: Allow Expert site: Allow What the profiles mean |
How to recognize it
You will not see it in logs. Apple says Applebot-Extended does not crawl webpages; it only changes how data Applebot collects may be used. Any request whose user agent contains Applebot-Extended is fake.
- robots.txt token:
Applebot-Extended - IP list: none, because no request carries the name.
- Reverse DNS: none, for the same reason.
What it does
Applebot-Extended is a robots.txt token that lets a site refuse the use of its content for training Apple’s general purpose foundation models, the ones behind Apple’s generative AI features. Applebot keeps crawling the site for search either way.
What you gain by allowing it
- Your content in what Apple’s models learn from.
What you lose
By allowing it
- Content used for training with no traffic or payment in return.
By blocking it
- Nothing in Apple’s search. Apple says disallowing the token does not remove a site from search results.
- Presence in Apple’s training data. A publisher that licenses content keeps leverage; an expert site gives up a channel.
How to verify it
There is nothing to verify: no real request uses this name. Our trackers mark every such request as a fake on arrival.
Gotchas and history
- Not a crawler: a firewall rule on the
Applebot-Extendeduser agent blocks only fakes. The decision lives in robots.txt. - Googlebot fallback: Apple documents the fallback to Googlebot rules for Applebot only. Do not count on a
Google-Extendedrule covering Apple; nameApplebot-Extendedexplicitly.
What we observed
Data: Cloudflare Logpush for two of our sites, 19-26 October 2026 (8 days).
- No request carried the name, not even from the Google Cloud scanner.
About Apple
- Applebot crawls for the search in Spotlight, Siri and Safari. It may also feed training of Apple’s foundation models. With no Applebot group in robots.txt, it follows the Googlebot group.
- Applebot-Extended is a robots.txt token, not a crawler. It decides whether Applebot’s data may train Apple’s general purpose foundation models, without affecting search.
One crawler, two decisions. To stay in Apple’s search but out of training, allow Applebot and disallow Applebot-Extended.