Critical severity

How do I fix PerplexityBot being blocked on my site?

PerplexityBot is blocked either by a robots.txt Disallow rule — directly or through a User-agent: * catch-all — or by a CDN/WAF returning 401/403 to its requests. Add an explicit Allow stanza for PerplexityBot, remove any firewall rule that rejects the user-agent, then re-test the URL as PerplexityBot.

Agents affected

Perplexity (PerplexityBot)

How to tell you have this problem

Our scanner reports "robots.txt blocks PerplexityBot" (check robots_txt_blocks_perplexitybot, critical) or a 403 for the PerplexityBot user-agent.
Perplexity never lists your site among its sources, even for your own brand name.
Your WAF logs show blocked requests with "PerplexityBot" in the user-agent string.

How to fix it

1
Add an explicit Allow group to robots.txt (this overrides a catch-all "User-agent: *" Disallow):
```
User-agent: PerplexityBot
Allow: /

User-agent: Perplexity-User
Allow: /
```

Test how your server responds to the bot user-agent — 200 is a pass, 401/403 means a CDN or WAF layer is still blocking it:

curl -s -o /dev/null -w "%{http_code}\n" \
  -A "Mozilla/5.0 (compatible; PerplexityBot/1.0; +https://perplexity.ai/perplexitybot)" \
  https://yoursite.com/

If you get 401/403, allowlist the bot in your CDN/WAF (see our Cloudflare guide) rather than only editing robots.txt — the firewall fires before robots.txt is ever considered.
Re-run a free scan; both the robots.txt check and the live bot-fetch check must pass.

FAQ

Why does Perplexity matter more than other AI crawlers for traffic?

Perplexity cites sources prominently with clickable links on every answer, so being crawlable translates directly into referral traffic. Blocking PerplexityBot removes you from those citations.

What is Perplexity-User?

It is the user-triggered fetcher: when someone asks Perplexity about a specific URL, Perplexity-User retrieves it live. Our robots.txt check focuses on PerplexityBot, but allowing both covers indexing and live lookups.

My robots.txt allows PerplexityBot but the scan still fails. Why?

The most common cause is a CDN bot-protection preset returning 403 before your origin responds. That is a different finding in our scanner (network/CDN-layer block) with a different fix — adjust the WAF, not robots.txt.

Not sure if your site has this issue?

Run a free scan — we test this check (and 60+ others) against your live site and give you the exact paste-ready fix. No signup needed.