# robots txt fetch marker across crawlers

robots txt fetch marker across 1 crawlers from 1 vendors, from 1 claims, last verified 2026-09-13.

robots txt fetch marker across 1 crawlers from 1 vendors, from 1 claims, last verified 2026-09-13.

| Crawler | Vendor | Value | Method | Verified | Claim |
| --- | --- | --- | --- | --- | --- |
| GPTBot | OpenAI | yes | vendor_doc | 2026-09-13 | claim 85 |

## Not documented

| Crawler | Vendor |
| --- | --- |
| Claude-SearchBot | Anthropic |
| Claude-User | Anthropic |
| ClaudeBot | Anthropic |
| Applebot | Apple |
| Applebot-Extended | Apple |
| Google-Extended | Google |
| GoogleOther | Google |
| Googlebot | Google |
| Bingbot | Microsoft |
| ChatGPT-User | OpenAI |
| OAI-AdsBot | OpenAI |
| OAI-SearchBot | OpenAI |
| Perplexity-User | Perplexity |
| PerplexityBot | Perplexity |

## Sources

| Source |
| --- |
| https://developers.openai.com/api/docs/bots |