Snapshots for 2026-09

2026-09, day by day.

This month is complete. Each day below lists its facts and a table of crawler requests. The numbers come from one website. They describe what these crawlers did here, not across the web.

This page is also available as JSON, Markdown and CSV. Cite as: Benes, Peter. Rattlesnakes By Mail: A Field Guide to the Crawlers of the Web. benesthemenace.com. Licence CC BY 4.0.

2026-09-13

F-260913-01 ClaudeBot requested this site for the first time on 2026-09-13; its first request was for /crawlers/claudebot. Source
F-260913-02 GPTBot requested this site for the first time on 2026-09-13; its first request was for /crawlers/gptbot. Source
F-260913-03 OAI-SearchBot requested this site for the first time on 2026-09-13; its first request was for /robots.txt. Source
F-260913-04 Every request on 2026-09-13 from OAI-SearchBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	1	0%	1	n/a	no
GPTBot	1	0%	1	n/a	no
OAI-SearchBot	1	100%	1	n/a	no

2026-09-14

F-260914-01 ClaudeBot made 535 requests on 2026-09-14, its highest daily count since the site launched on 2026-09-13; its previous high was 1, on 2026-09-13. Source
F-260914-02 GPTBot made 3 requests on 2026-09-14, its highest daily count since the site launched on 2026-09-13; its previous high was 1, on 2026-09-13. Source
F-260914-03 OAI-SearchBot made 2 requests on 2026-09-14, its highest daily count since the site launched on 2026-09-13; its previous high was 1, on 2026-09-13. Source
F-260914-04 Googlebot requested this site for the first time on 2026-09-14; its first request was for /robots.txt. Source
F-260914-05 GoogleOther requested this site for the first time on 2026-09-14; its first request was for /. Source
F-260914-06 Applebot requested this site for the first time on 2026-09-14; its first request was for /robots.txt. Source
F-260914-07 ClaudeBot requested a CSV file for the first time on 2026-09-14. Source
F-260914-08 ClaudeBot requested a JSON file for the first time on 2026-09-14. Source
F-260914-09 ClaudeBot requested a Markdown file for the first time on 2026-09-14. Source
F-260914-10 ClaudeBot requested a text file for the first time on 2026-09-14. Source
F-260914-11 ClaudeBot requested an XML file for the first time on 2026-09-14. Source
F-260914-12 Every request on 2026-09-14 from GoogleOther, Applebot and OAI-SearchBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	535	99.8%	528	1	no
Googlebot	79	98.7%	43	0	no
GoogleOther	76	100%	65	0	no
Applebot	6	100%	2	0	no
GPTBot	3	0%	1	1	no
OAI-SearchBot	2	100%	1	1	no

2026-09-15

F-260915-01 Googlebot made 133 requests on 2026-09-15, its highest daily count since the site launched on 2026-09-13; its previous high was 79, on 2026-09-14. Source
F-260915-02 GoogleOther made 121 requests on 2026-09-15, its highest daily count since the site launched on 2026-09-13; its previous high was 76, on 2026-09-14. Source
F-260915-03 Claude-User requested this site for the first time on 2026-09-15; its first request was for /robots.txt. Source
F-260915-04 Bingbot requested this site for the first time on 2026-09-15; its first request was for /. Source
F-260915-05 GoogleOther requested a CSV file for the first time on 2026-09-15. Source
F-260915-06 Claude-User fetched 5 pages on 2026-09-15 in response to people's requests; the most fetched was /robots.txt. Source
F-260915-07 Every request on 2026-09-15 from Googlebot, GoogleOther, ClaudeBot, Claude-User and Bingbot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Googlebot	133	100%	88	40	no
GoogleOther	121	100%	106	38	no
ClaudeBot	24	100%	2	268	no
Claude-User	5	100%	4	0	no
Bingbot	4	100%	4	0	no
Applebot	2	100%	1	3	no

2026-09-16

F-260916-01 GPTBot made 666 requests on 2026-09-16, its highest daily count since the site launched on 2026-09-13; its previous high was 3, on 2026-09-14. Source
F-260916-02 GPTBot requested a CSV file for the first time on 2026-09-16. Source
F-260916-03 GPTBot requested a JSON file for the first time on 2026-09-16. Source
F-260916-04 GPTBot requested a Markdown file for the first time on 2026-09-16. Source
F-260916-05 GPTBot requested an XML file for the first time on 2026-09-16. Source
F-260916-06 Googlebot requested a CSV file for the first time on 2026-09-16. Source
F-260916-07 Every request on 2026-09-16 from GPTBot, GoogleOther, Googlebot, ClaudeBot and OAI-SearchBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
GPTBot	666	100%	653	1	no
GoogleOther	46	100%	41	76	no
Googlebot	30	100%	17	79	no
ClaudeBot	24	100%	2	24	no
OAI-SearchBot	2	100%	1	1	no

2026-09-17

F-260917-01 Applebot made 19 requests on 2026-09-17, its highest daily count since the site launched on 2026-09-13; its previous high was 6, on 2026-09-14. Source
F-260917-02 Claude-User requested a JSON file for the first time on 2026-09-17. Source
F-260917-03 Claude-User requested a Markdown file for the first time on 2026-09-17. Source
F-260917-04 Claude-User fetched /.well-known/mcp/server.json on 2026-09-17; it is the first crawler to do so here. Source
F-260917-05 Claude-User fetched 5 pages on 2026-09-17 in response to people's requests; the most fetched was /.well-known/mcp/server.json. Source
F-260917-06 Every request on 2026-09-17 from ClaudeBot, Applebot, GoogleOther, Claude-User and Googlebot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	24	100%	2	24	no
Applebot	19	100%	17	1	no
GoogleOther	8	100%	8	61	no
Claude-User	5	100%	5	0	no
Googlebot	4	100%	1	55	no

2026-09-18

F-260918-01 GoogleOther made 206 requests on 2026-09-18, its highest daily count since the site launched on 2026-09-13; its previous high was 121, on 2026-09-15. Source
F-260918-02 OAI-SearchBot made 3 requests on 2026-09-18, its highest daily count since the site launched on 2026-09-13; its previous high was 2, on 2026-09-14. Source
F-260918-03 ChatGPT-User requested this site for the first time on 2026-09-18; its first request was for /mcp. Source
F-260918-04 Applebot requested a JSON file for the first time on 2026-09-18. Source
F-260918-05 Applebot requested a Markdown file for the first time on 2026-09-18. Source
F-260918-06 ChatGPT-User fetched 2 pages on 2026-09-18 in response to people's requests; the most fetched was /. Source
F-260918-07 Claude-User fetched 2 pages on 2026-09-18 in response to people's requests; the most fetched was /. Source
F-260918-08 Every request on 2026-09-18 from ClaudeBot, GPTBot, GoogleOther, Googlebot and Applebot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	469	100%	438	24	no
GPTBot	436	100%	434	1	no
GoogleOther	206	100%	174	46	no
Googlebot	78	100%	35	30	no
Applebot	9	100%	8	2	no
OAI-SearchBot	3	100%	1	1	no
ChatGPT-User	2	100%	2	0	no
Claude-User	2	100%	2	0	no

2026-09-19

F-260919-01 ChatGPT-User made 3 requests on 2026-09-19, its highest daily count since the site launched on 2026-09-13; its previous high was 2, on 2026-09-18. Source
F-260919-02 ChatGPT-User fetched 3 pages on 2026-09-19 in response to people's requests; the most fetched was /compare/applebot-extended-vs-oai-adsbot. Source
F-260919-03 Claude-User fetched 5 pages on 2026-09-19 in response to people's requests; the most fetched was /robots.txt. Source
F-260919-04 An MCP client named verifymcp-probe first called __verifymcp_auth_probe_3d0d2b52443f4ac0__ on 2026-09-19. Source
F-260919-05 Every request on 2026-09-19 from ClaudeBot, GoogleOther, Googlebot, Claude-User and ChatGPT-User was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	31	100%	4	24	no
GoogleOther	13	100%	12	61	no
Googlebot	7	100%	2	54	no
Claude-User	5	100%	4	1	no
ChatGPT-User	3	100%	3	0	no
OAI-SearchBot	2	100%	1	2	no

2026-09-20

F-260920-01 Applebot made 171 requests on 2026-09-20, its highest daily count since the site launched on 2026-09-13; its previous high was 19, on 2026-09-17. Source
F-260920-02 OAI-SearchBot made 4 requests on 2026-09-20, its highest daily count since the site launched on 2026-09-13; its previous high was 3, on 2026-09-18. Source
F-260920-03 Applebot made 171 requests on 2026-09-20, 85.5 times its 28-day median of 2. Source
F-260920-04 Applebot requested a CSV file for the first time on 2026-09-20. Source
F-260920-05 Claude-User fetched /mcp on 2026-09-20; it is the first crawler to do so here. Source
F-260920-06 An MCP client named sasame-audit first called lookup_crawler on 2026-09-20. Source
F-260920-07 An MCP client named rokmcp-probe first called list_changes on 2026-09-20. Source
F-260920-08 An MCP client named Smithery Connect first called list_changes on 2026-09-20. Source
F-260920-09 An MCP client named mcphub-probe first called list_changes on 2026-09-20. Source
F-260920-10 An MCP client named brick.blue first called list_changes on 2026-09-20. Source
F-260920-11 Every request on 2026-09-20 from Applebot, Googlebot, OAI-SearchBot, GPTBot and ClaudeBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Applebot	171	100%	166	2	yes
Googlebot	10	100%	5	30	yes
OAI-SearchBot	4	100%	1	2	no
GPTBot	3	100%	3	1	no
Claude-User	2	0%	1	2	no
ClaudeBot	2	100%	2	24	no

2026-09-21

F-260921-01 Googlebot re-read the whole site on 2026-09-21 (241 requests); this is the first full re-read recorded here. Source
F-260921-02 Googlebot made 711 requests on 2026-09-21, its highest daily count since the site launched on 2026-09-13; its previous high was 133, on 2026-09-15. Source
F-260921-03 Googlebot made 711 requests on 2026-09-21, 35.6 times its 28-day median of 20. Source
F-260921-04 Applebot made 148 requests on 2026-09-21, 37 times its 28-day median of 4. Source
F-260921-05 Googlebot requested a Markdown file for the first time on 2026-09-21. Source
F-260921-06 Applebot requested an XML file for the first time on 2026-09-21. Source
F-260921-07 Every request on 2026-09-21 from Applebot, ClaudeBot, Bingbot, OAI-SearchBot and GoogleOther was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Googlebot	711	2.3%	691	20	yes
Applebot	148	100%	140	4	yes
ClaudeBot	28	100%	5	24	no
Bingbot	3	100%	2	0	no
OAI-SearchBot	2	100%	1	2	no
GoogleOther	1	100%	1	30	no

2026-09-22

F-260922-01 Applebot re-read the whole site on 2026-09-22 (320 requests); this is the first full re-read recorded here. Source
F-260922-02 OAI-SearchBot made 5 requests on 2026-09-22, its highest daily count since the site launched on 2026-09-13; its previous high was 4, on 2026-09-20. Source
F-260922-03 Every request on 2026-09-22 from Applebot, Googlebot, OAI-SearchBot, ClaudeBot and Bingbot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Applebot	11	100%	11	6	yes
Googlebot	11	100%	3	30	no
OAI-SearchBot	5	100%	1	2	no
ClaudeBot	4	100%	4	24	no
Bingbot	2	100%	2	0	no

2026-09-23

F-260923-01 Applebot made 151 requests on 2026-09-23, 20.1 times its 28-day median of 8. Source
F-260923-02 ChatGPT-User fetched 1 page on 2026-09-23 in response to people's requests; the most fetched was /claims/85. Source
F-260923-03 Every request on 2026-09-23 from Applebot, Googlebot, ClaudeBot, Bingbot and OAI-SearchBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Applebot	151	100%	140	8	no
Googlebot	28	100%	7	21	no
ClaudeBot	16	100%	4	24	no
Bingbot	2	100%	2	0	no
OAI-SearchBot	2	100%	1	2	no
ChatGPT-User	1	100%	1	0	no

2026-09-24

F-260924-01 ClaudeBot re-read the whole site on 2026-09-24 (448 requests); this is the first full re-read recorded here. Source
F-260924-02 GPTBot re-read the whole site on 2026-09-24 (462 requests); this is the first full re-read recorded here. Source
F-260924-03 ClaudeBot made 1,141 requests on 2026-09-24, its highest daily count since the site launched on 2026-09-13; its previous high was 535, on 2026-09-14. Source
F-260924-04 GPTBot made 1,101 requests on 2026-09-24, its highest daily count since the site launched on 2026-09-13; its previous high was 666, on 2026-09-16. Source
F-260924-05 Applebot made 426 requests on 2026-09-24, its highest daily count since the site launched on 2026-09-13; its previous high was 171, on 2026-09-20. Source
F-260924-06 ClaudeBot made 1,141 requests on 2026-09-24, 47.5 times its 28-day median of 24. Source
F-260924-07 Applebot made 426 requests on 2026-09-24, 47.3 times its 28-day median of 9. Source
F-260924-08 An MCP client named mcp-vouch first called list_changes on 2026-09-24. Source
F-260924-09 Every request on 2026-09-24 from ClaudeBot, GPTBot, Applebot, Googlebot and OAI-SearchBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	1,141	100%	1,091	24	yes
GPTBot	1,101	100%	1,091	0	yes
Applebot	426	100%	351	9	no
Googlebot	9	100%	3	28	no
OAI-SearchBot	5	100%	1	2	no
Bingbot	2	100%	2	0	no

2026-09-25

F-260925-01 Applebot made 62 requests on 2026-09-25, 6.2 times its 28-day median of 10. Source
F-260925-02 PerplexityBot requested this site for the first time on 2026-09-25; its first request was for /robots.txt. Source
F-260925-03 ChatGPT-User fetched 1 page on 2026-09-25 in response to people's requests; the most fetched was /compare/gptbot-vs-perplexity-user. Source
F-260925-04 Every request on 2026-09-25 from Applebot, ClaudeBot, Bingbot, OAI-SearchBot and GPTBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Applebot	62	100%	56	10	no
ClaudeBot	55	100%	4	24	no
Googlebot	21	95.2%	6	20	no
Bingbot	4	100%	3	0	no
OAI-SearchBot	4	100%	1	2	no
GPTBot	3	100%	3	1	no
GoogleOther	2	100%	2	5	no
PerplexityBot	2	100%	2	0	no
ChatGPT-User	1	100%	1	0	no

2026-09-26

F-260926-01 An MCP client named katalir-readonly-verifier first called list_changes on 2026-09-26. Source
F-260926-02 Every request on 2026-09-26 from ClaudeBot, Applebot, Googlebot, Bingbot and GPTBot was verified against the operator's published method. Source
F-260926-03 7 crawlers made 87 requests on 2026-09-26; ClaudeBot made the most, with 48. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	48	100%	4	24	no
Applebot	16	100%	14	11	no
Googlebot	12	100%	7	21	no
Bingbot	3	100%	2	0	no
GPTBot	3	100%	3	1	no
OAI-SearchBot	3	100%	1	2	no
GoogleOther	2	100%	2	2	no

2026-09-27

F-260927-01 Googlebot made 84 requests on 2026-09-27, 5.1 times its 28-day median of 17. Source
F-260927-02 GoogleOther made 80 requests on 2026-09-27, 40 times its 28-day median of 2. Source
F-260927-03 GPTBot made 61 requests on 2026-09-27, 30.5 times its 28-day median of 2. Source
F-260927-04 Applebot made 47 requests on 2026-09-27, 3.5 times its 28-day median of 14. Source
F-260927-05 Claude-User has made no requests since 2026-09-20, 7 days. Source
F-260927-06 Googlebot requested a JSON file for the first time on 2026-09-27. Source
F-260927-07 ChatGPT-User fetched 3 pages on 2026-09-27 in response to people's requests; the most fetched was /crawlers/oai-searchbot. Source
F-260927-08 An MCP client named mcp-probe first called list_changes on 2026-09-27. Source
F-260927-09 An MCP client named agent-scout first called list_changes on 2026-09-27. Source
F-260927-10 Every request on 2026-09-27 from Googlebot, GoogleOther, GPTBot, Applebot and ClaudeBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
Googlebot	84	100%	56	17	no
GoogleOther	80	100%	54	2	no
GPTBot	61	100%	57	2	no
Applebot	47	100%	36	14	no
ClaudeBot	47	100%	4	26	no
OAI-SearchBot	5	100%	1	2	no
ChatGPT-User	3	100%	3	0	no
Bingbot	2	100%	2	1	no

2026-09-28

F-260928-01 Every request on 2026-09-28 from ClaudeBot, Applebot, Googlebot, GPTBot and OAI-SearchBot was verified against the operator's published method. Source
F-260928-02 6 crawlers made 73 requests on 2026-09-28; ClaudeBot made the most, with 49. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	49	100%	4	28	no
Applebot	8	100%	8	16	no
Googlebot	8	100%	5	21	no
GPTBot	3	100%	3	3	no
OAI-SearchBot	3	100%	1	2	no
Bingbot	2	100%	1	2	no

2026-09-29

F-260929-01 Every request on 2026-09-29 from ClaudeBot, GPTBot, OAI-SearchBot, Applebot and Googlebot was verified against the operator's published method. Source
F-260929-02 5 crawlers made 59 requests on 2026-09-29; ClaudeBot made the most, with 48. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	48	100%	4	30	no
GPTBot	4	100%	4	3	no
OAI-SearchBot	3	100%	1	3	no
Applebot	2	100%	2	14	no
Googlebot	2	100%	2	17	no

2026-09-30

F-260930-01 Bingbot made 16 requests on 2026-09-30, its highest daily count since the site launched on 2026-09-13; its previous high was 4, on 2026-09-15. Source
F-260930-02 Bingbot made 16 requests on 2026-09-30, 8 times its 28-day median of 2. Source
F-260930-03 Every request on 2026-09-30 from ClaudeBot, Bingbot, Googlebot, GPTBot and OAI-SearchBot was verified against the operator's published method. Source

Crawler	Requests	Verified share	Distinct paths	28-day median	Sweep day
ClaudeBot	46	100%	4	31	no
Bingbot	16	100%	14	2	no
Googlebot	7	100%	5	12	no
GPTBot	3	100%	3	3	no
OAI-SearchBot	3	100%	1	3	no

All snapshots: https://rattlesnakesbymail.com/snapshots