Anthropic ClaudeBot
AI training
About this crawler
Collects pages that may train Anthropic models.
Your page is being collected, and it may be used to build AI models. No person is waiting on this visit.
User agent
ClaudeBot
DataFair counts a request as ClaudeBot when its user agent contains that name, in any letter case. Requests like this stay out of your visitors and your monthly allowance.
IP addresses
Anthropic publishes the addresses this crawler uses. DataFair does not compare requests against them yet, so a count means a request used this name, not proof that Anthropic sent it.
https://claude.com/crawling/bots.json
Control it in robots.txt
User-agent: ClaudeBot Disallow: /
This asks the crawler to stay away. Compliance is up to the operator.