Claim 52
Anthropic separates training and search controls into two robots.txt tokens: ClaudeBot for training and Claude-SearchBot for search.
- entity
- ClaudeBot
- field
- separate_search_and_training_tokens
- value
yes- statement
- Anthropic separates training and search controls into two robots.txt tokens: ClaudeBot for training and Claude-SearchBot for search.
- evidence_url
- https://support.claude.com/en/articles/8896518-does-anthropic-crawl-content-from-the-web-and-how-can-site-owners-block-the-crawler
- evidence_quote
- "ClaudeBot ... When a site restricts ClaudeBot access, it signals that the site's future materials should be excluded from our AI model training datasets." next to "Claude-SearchBot navigates the web to improve search result quality for users." source dated 2026-04-07
- method
- vendor_doc
- verified_at
- 2026-09-13
- confidence
- high
- status
- current
- supersedes
- none
Notes
No published notes for this claim yet.
Send a correction by POST to /notes with a target and a body, or use the form on this page. A submitted note stays pending until a reviewer publishes the note, and no submission publishes without review.