Fix your AI visibility issues
Step-by-step guides for the issues our scanner finds most often — each with a direct answer, symptoms, and a paste-ready fix. These are the same checks the free scan runs against your live site.
How to fix: GPTBot blocked by robots.txt
GPTBot gets blocked when robots.txt has a Disallow rule under User-agent: GPTBot, or a blanket User-agent: * Disallow with no exception. Remove that rule or add an explicit Allow stanza for GPTBot, redeploy the file at /robots.txt, then re-scan to confirm OpenAI crawlers can read the page.
How to fix: ClaudeBot blocked by robots.txt
ClaudeBot is blocked when robots.txt contains a Disallow rule that matches Anthropic crawlers — either under User-agent: ClaudeBot or under a catch-all User-agent: * group. Delete the Disallow line or add an Allow stanza for ClaudeBot, plus Claude-User and Claude-Web for live browsing, then redeploy and re-scan.
How to fix: PerplexityBot blocked
PerplexityBot is blocked either by a robots.txt Disallow rule — directly or through a User-agent: * catch-all — or by a CDN/WAF returning 401/403 to its requests. Add an explicit Allow stanza for PerplexityBot, remove any firewall rule that rejects the user-agent, then re-test the URL as PerplexityBot.
How to fix: Cloudflare blocking AI crawlers
AI crawlers receive 401 or 403 from Cloudflare when Bot Fight Mode, Super Bot Fight Mode, or the "Block AI bots" setting rejects them before your application responds. Fix it by disabling the AI-bot block under Security, then adding a WAF custom rule that skips bot protection for the AI crawlers you want.
How to fix: Page is blank without JavaScript
AI crawlers like GPTBot and ClaudeBot do not execute JavaScript, so a client-side rendered single-page app looks like an empty page to them. Fix it with server-side rendering or prerendering — Next.js, Nuxt, Remix, Astro, or a prerender service — so the raw HTML already contains your headline, copy, and links.
How to fix: X-Robots-Tag noindex blocking AI
A response header like X-Robots-Tag: noindex — or the AI-specific noai directive — tells crawlers to drop the page entirely, removing it from both classic search and AI answers. Find where the header is set (web server config, CDN rule, or application middleware), delete the directive, then confirm with curl -I.
How to fix: Missing Product schema for AI shopping
AI shopping surfaces in ChatGPT, Perplexity, and Google read Product JSON-LD to get your price, availability, and images — without it, product pages are effectively invisible to them. Add a Product block with name, image, description, brand, and a complete Offer including price, priceCurrency, and availability, then validate.
How to fix: Missing FAQ schema (FAQPage JSON-LD)
If a page has question-style headings but no FAQPage JSON-LD, AI assistants and search engines must guess the question-answer structure. Add a single JSON-LD block with @type FAQPage listing every visible question and its complete answer, keep it in sync with the on-page text, then validate it.
How to fix: Missing Organization schema
When a site declares JSON-LD but never an Organization entity, AI assistants cannot reliably tell who is behind the domain. Add one Organization JSON-LD block on your homepage with name, url, logo, and sameAs links to your official profiles, then validate it with the schema.org validator and re-scan.
How to fix: llms.txt missing or invalid
llms.txt is an optional, emerging convention from llmstxt.org — Google has said it does not use it, and evidence of AI-crawler pickup is still limited, so treat it as a low-cost bonus, not a requirement. If you add one, start with a single H1 title, a blockquote summary, and H2 link sections.