Crawler directory

Anthropic Claude-User

AI answers

About this crawler

Opens a page while Claude answers someone.

Someone asked an AI assistant a question, and the assistant opened your page to answer. These visits show which pages AI tools are using right now.

User agent

Claude-User

DataFair counts a request as Claude-User when its user agent contains that name, in any letter case. Requests like this stay out of your visitors and your monthly allowance.

IP addresses

Anthropic publishes the addresses this crawler uses. DataFair does not compare requests against them yet, so a count means a request used this name, not proof that Anthropic sent it.

https://claude.com/crawling/bots.json

Control it in robots.txt

User-agent: Claude-User
Disallow: /

This asks the crawler to stay away. Blocking it can also keep you out of its answers or results, so decide with care. Fetches a person triggers may not follow robots.txt, so check the operator's documentation.

Sources

Anthropic documentation

More from Anthropic

Track crawlers with DataFair