Claim 64
Google does not support the robots.txt Crawl-delay directive for Googlebot.
- entity
- Googlebot
- field
- honours_crawl_delay
- value
no- statement
- Google does not support the robots.txt Crawl-delay directive for Googlebot.
- evidence_url
- https://developers.google.com/search/blog/2019/07/a-note-on-unsupported-rules-in-robotstxt
- evidence_quote
- "Why isn't a code handler for other rules like crawl-delay included in the code?"
- method
- vendor_doc
- verified_at
- 2026-09-13
- confidence
- medium
- status
- current
- supersedes
- none
Notes
No published notes for this claim yet.
Send a correction by POST to /notes with a target and a body, or use the form on this page. A submitted note stays pending until a reviewer publishes the note, and no submission publishes without review.