Rattlesnakes By Mail

Claude-User vs Google-Extended

Claude-User and Google-Extended compared on 12 documented fields, from 18 claims, last verified 2026-09-13.

Field	Claude-User	Google-Extended	Comparison
cites_sources	not_documented	not documented	different
honours_crawl_delay	yes	n/a	different
publishes_ip_list	yes	no	different
purpose_documented	user_fetch	policy_only	different
reads_llms_txt	not_documented	not documented	different
respects_robots_txt	yes	n/a	different
robots_token_differs_from_ua	not documented	yes	different
separate_search_and_training_tokens	not_documented	yes	different
user_agent_string_full	not_documented	not documented	different
user_fetch_exception_documented	no	not documented	different
verifiable_by_rdns	not_documented	n/a	different
what_it_controls	not documented	Gemini model training and grounding, not Search inclusion	different

Differences

Field	Claude-User statement	Google-Extended statement
cites_sources	Anthropic does not publish whether Claude responses cite pages fetched by Claude-User.	not documented
honours_crawl_delay	Claude-User respects the Crawl-delay directive in robots.txt when fetching pages.	Google does not publish any Crawl-delay behaviour for the Google-Extended token.
publishes_ip_list	Anthropic publishes the IP address ranges used by Claude-User as a JSON file at https://claude.com/crawling/bots.json.	Google publishes no IP list for Google-Extended because Google-Extended has no separate HTTP request user agent string.
purpose_documented	Claude-User accesses websites in response to questions that individual users ask Claude.	Google-Extended is a standalone product token that controls whether Google uses crawled content to train Gemini models.
reads_llms_txt	Anthropic does not publish whether Claude-User reads llms.txt files.	not documented
respects_robots_txt	Claude-User honours industry standard robots.txt directives that signal do not crawl.	Google-Extended is a robots.txt token that publishers set to control use of crawled content for Gemini training.
robots_token_differs_from_ua	not documented	Google-Extended is a robots.txt control token and does not correspond to a crawler user agent string.
separate_search_and_training_tokens	Anthropic does not document Claude-User as a search or training opt-out token.	Google-Extended controls Gemini training use only and does not affect inclusion in Google Search.
user_agent_string_full	Anthropic does not publish a full user agent string for Claude-User.	not documented
user_fetch_exception_documented	Anthropic applies the same robots.txt statement to Claude-User as to ClaudeBot and Claude-SearchBot and documents no exception for user-initiated fetches.	not documented
verifiable_by_rdns	Anthropic does not publish a reverse-DNS verification method for Claude-User.	Google does not publish a reverse-DNS verification method for Google-Extended.
what_it_controls	not documented	Google-Extended controls AI training and grounding in Google systems and does not control Google Search inclusion.

Sources