Crawler directory

Applebot-Extended

AI training

About this crawler

A robots.txt name that controls whether Apple may use your pages to train its models.

You won't see this name in your traffic. Apple reads it from your robots.txt to decide how your pages may be used, and its other crawlers do the fetching.

User agent

Applebot-Extended

DataFair never counts this name, because no request carries it.

IP addresses

This name has no addresses, because no request carries it.

Control it in robots.txt

User-agent: Applebot-Extended
Disallow: /

This asks the crawler to stay away. Compliance is up to the operator.

Sources

Apple documentation

More from Apple

Track crawlers with DataFair