Will Cloudflare auto-block AI crawlers on your site?
From September 15, 2026, Cloudflare blocks mixed-use AI crawlers by default on ad-monetized pages. If your site is behind Cloudflare, runs ads, and you want ChatGPT or Perplexity citing you — you may be blocked without noticing. Check it in 60 seconds, free.
You are in the blast radius if all three are true
Our scanner detects Cloudflare, checks ad signals (including ads.txt) and reports the exposure per AI agent — alongside the rest of the AI-readiness audit.
FAQ
What happens on September 15, 2026?
Cloudflare starts blocking "mixed-use" AI crawlers — bots that combine search indexing with AI training or agent use — by default on pages that show ads, across sites on its network. AI companies were asked to separate crawlers by purpose; bots that do not comply get the default block on monetized pages.
My robots.txt allows GPTBot. Am I safe?
Not necessarily. The block happens at Cloudflare’s edge before the crawler reaches your origin, so robots.txt allow rules cannot override it. The switch that matters is in your Cloudflare dashboard, under AI Crawl Control.
How do I check if my site is affected?
Run our free scan: it fetches your site as GPTBot, ClaudeBot, PerplexityBot, Google-Extended, Doubao and Grok, detects Cloudflare, checks for ad signals like ads.txt, and flags the September 15 exposure with per-agent impact — in about 60 seconds, no signup.
What if I want to block or charge AI bots instead?
Then the new default works in your favor. Decide per bot, deliberately: blocking OAI-SearchBot removes you from ChatGPT search, while blocking GPTBot only affects model training. Cloudflare’s Pay Per Use program and per-URL bot paywalls (which we support on paid plans) let you monetize instead of just refusing.