# AI crawler pacing. Added 2026-08-10.
#
# DO NOT add Crawl-delay to the "User-agent: *" group. A global
# Crawl-delay: 10 was flagged in the 2026-06-07 top1.fm audit as throttling
# indexing of 800+ episodes, and was removed on 2026-06-08. Google ignores
# Crawl-delay; Bing honors it, so a global value costs Bing indexation.
# Any pacing here stays scoped to named agents.
#
# GPTBot and ClaudeBot receive HTTP 429 from the WP Engine origin rate
# limiter (WPE case 8578215). Anthropic documents Crawl-delay support for
# ClaudeBot; OpenAI does not document it for GPTBot, so that group may be
# ignored. Revisit once WP Engine confirms the real threshold.

User-agent: ClaudeBot
Disallow:
Crawl-delay: 100

User-agent: GPTBot
Disallow:
Crawl-delay: 100

User-agent: *
Disallow:

Sitemap: https://top1.fm/sitemap_index.xml
Sitemap: https://top1.fm/DailySalesTips/sitemap_index.xml