Rattlesnakes By Mail

reads llms txt across crawlers

reads llms txt across 9 crawlers from 6 vendors, from 9 claims, last verified 2026-09-14.

reads llms txt across 9 crawlers from 6 vendors, from 9 claims, last verified 2026-09-14.

CrawlerVendorValueMethodVerifiedClaim
ApplebotApplenot_documentedvendor_doc2026-09-13claim 16
BingbotMicrosoftnot_documentedvendor_doc2026-09-14claim 128
ChatGPT-UserOpenAInot_documentedvendor_doc2026-09-13claim 26
Claude-SearchBotAnthropicnot_documentedvendor_doc2026-09-13claim 35
Claude-UserAnthropicnot_documentedvendor_doc2026-09-13claim 45
GooglebotGooglenovendor_doc2026-09-14claim 132
OAI-SearchBotOpenAInot_documentedvendor_doc2026-09-13claim 101
Perplexity-UserPerplexitynot_documentedvendor_doc2026-09-13claim 110
PerplexityBotPerplexitynot_documentedvendor_doc2026-09-13claim 119

Not documented

CrawlerVendor
ClaudeBotAnthropic
Applebot-ExtendedApple
Google-ExtendedGoogle
GoogleOtherGoogle
GPTBotOpenAI
OAI-AdsBotOpenAI

Sources

Source
https://support.apple.com/en-us/119829
https://developers.openai.com/api/docs/bots
https://support.claude.com/en/articles/8896518-does-anthropic-crawl-content-from-the-web-and-how-can-site-owners-block-the-crawler
https://docs.perplexity.ai/docs/resources/perplexity-crawlers
https://www.bing.com/webmasters/help/which-crawlers-does-bing-use-8c184ec0
https://developers.google.com/search/docs/fundamentals/ai-optimization-guide