Rattlesnakes By Mail

ClaudeBot vs Google-Extended

ClaudeBot and Google-Extended compared on 14 documented fields, from 20 claims, last verified 2026-09-15.

FieldClaudeBotGoogle-ExtendedComparison
anti_circumvention_stancedocumentednot documenteddifferent
honours_crawl_delayyesn/adifferent
observed_fetches_jsonyesnot documenteddifferent
observed_fetches_markdownyesnot documenteddifferent
observed_first_request_path/sitemap.xmlnot documenteddifferent
observed_format_pass_orderhtml_json_mdnot documenteddifferent
publishes_ip_listyesnodifferent
purpose_documentedtrainingpolicy_onlydifferent
respects_robots_txtyesn/adifferent
robots_token_differs_from_uanot documentedyesdifferent
separate_search_and_training_tokensyesyessame
user_agent_string_fullnot_documentednot documenteddifferent
verifiable_by_rdnsnot_documentedn/adifferent
what_it_controlsnot documentedGemini model training and grounding, not Search inclusiondifferent

Differences

FieldClaudeBot statementGoogle-Extended statement
anti_circumvention_stanceClaudeBot respects anti-circumvention technologies and does not bypass CAPTCHAs on crawled sites.not documented
honours_crawl_delayAnthropic supports the non-standard robots.txt Crawl-delay extension for ClaudeBot.Google does not publish any Crawl-delay behaviour for the Google-Extended token.
observed_fetches_jsonClaudeBot made 177 JSON requests to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z, covering the .json twin of 170 base paths. The .json URLs of Rattlesnakes By Mail appear in no entry of the 171 URL sitemap.not documented
observed_fetches_markdownClaudeBot fetched the .md twin of 170 base paths on Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z, 170 Markdown requests of 542 ClaudeBot requests in that window. The .md URLs of Rattlesnakes By Mail appear in no entry of the 171 URL sitemap.not documented
observed_first_request_pathClaudeBot's first request to Rattlesnakes By Mail between 2026-09-13T20:00Z and 2026-09-15T09:00Z was /sitemap.xml at 2026-09-14T19:53:45Z. ClaudeBot fetched /robots.txt on the second request to Rattlesnakes By Mail.not documented
observed_format_pass_orderClaudeBot fetched Rattlesnakes By Mail in three format passes between 2026-09-13T20:00Z and 2026-09-15T09:00Z, HTML from 2026-09-14T20:52Z to 2026-09-14T20:57Z, then JSON from 2026-09-14T21:37Z to 2026-09-14T21:56Z, then Markdown from 2026-09-14T21:37Z to 2026-09-14T21:57Z. The same order holds on 170 of 170 base paths that ClaudeBot fetched in all three formats on Rattlesnakes By Mail.not documented
publishes_ip_listAnthropic publishes the IP address ranges used by ClaudeBot as a JSON file at https://claude.com/crawling/bots.json.Google publishes no IP list for Google-Extended because Google-Extended has no separate HTTP request user agent string.
purpose_documentedClaudeBot collects web content for training Anthropic generative AI models.Google-Extended is a standalone product token that controls whether Google uses crawled content to train Gemini models.
respects_robots_txtClaudeBot honours industry standard robots.txt directives that signal do not crawl.Google-Extended is a robots.txt token that publishers set to control use of crawled content for Gemini training.
robots_token_differs_from_uanot documentedGoogle-Extended is a robots.txt control token and does not correspond to a crawler user agent string.
user_agent_string_fullAnthropic does not publish a full user agent string for ClaudeBot.not documented
verifiable_by_rdnsAnthropic does not publish a reverse-DNS verification method for ClaudeBot.Google does not publish a reverse-DNS verification method for Google-Extended.
what_it_controlsnot documentedGoogle-Extended controls AI training and grounding in Google systems and does not control Google Search inclusion.

Sources