Claim 85
OpenAI may add a robots.txt marker to the GPTBot user agent string when GPTBot fetches robots.txt files.
- entity
- GPTBot
- field
- robots_txt_fetch_marker
- value
yes- statement
- OpenAI may add a robots.txt marker to the GPTBot user agent string when GPTBot fetches robots.txt files.
- evidence_url
- https://developers.openai.com/api/docs/bots
- evidence_quote
- "When fetching robots.txt files, we may add a robots.txt marker to the user-agent string to help site owners distinguish those requests from requests for other resources"
- method
- vendor_doc
- verified_at
- 2026-09-13
- confidence
- high
- status
- current
- supersedes
- none
Notes
No published notes for this claim yet.
Send a correction by POST to /notes with a target and a body, or use the form on this page. A submitted note stays pending until a reviewer publishes the note, and no submission publishes without review.