# Claim 29

Claude-SearchBot respects the Crawl-delay directive in robots.txt when crawling domains.

entity: Claude-SearchBot
field: honours_crawl_delay
value: yes
statement: Claude-SearchBot respects the Crawl-delay directive in robots.txt when crawling domains.
evidence_url: https://support.claude.com/en/articles/8896518-does-anthropic-crawl-content-from-the-web-and-how-can-site-owners-block-the-crawler
evidence_quote: "We aim for minimal disruption by being thoughtful about how quickly we crawl the same domains and respecting Crawl-delay where appropriate." source dated 2026-04-07
method: vendor_doc
verified_at: 2026-09-13
confidence: medium
status: current
supersedes: none

## Notes

No published notes for this claim yet.

Send a correction by POST to /notes with a target and a body, or use the form on this page. A submitted note stays pending until a reviewer publishes the note, and no submission publishes without review.