Info severity

Do I need an llms.txt file, and how do I write a valid one?

llms.txt is an optional, emerging convention from llmstxt.org — Google has said it does not use it, and evidence of AI-crawler pickup is still limited, so treat it as a low-cost bonus, not a requirement. If you add one, start with a single H1 title, a blockquote summary, and H2 link sections.

Agents affected

ClaudeChatGPTPerplexity

How to tell you have this problem

  • Our scanner reports "No /llms.txt file" (check missing_llms_txt — info severity, deliberately the lowest, because the standard is unproven).
  • Or the file exists but is malformed: it does not start with a Markdown H1, has multiple H1s, or its H2 sections contain no [name](url) link bullets.

How to fix it

  1. Decide if it is worth it — honestly. Google does not read llms.txt. Anthropic, OpenAI, and Perplexity have signaled they read it when present, but there is no public evidence it changes rankings or citations today. It costs ten minutes and cannot hurt, which is exactly why it is our lowest severity finding.

  2. If you add one, follow the llmstxt.org spec: exactly one H1 (your site name) as the first non-blank line, an optional blockquote summary, then H2 sections whose bullets are [name](url) links. Serve it at /llms.txt as plain text:

    # ACME Inc.

> ACME makes industrial-grade anvils and ships worldwide. The most useful pages for understanding our products and docs are below.

## Products
- [Anvil catalog](https://www.acme.com/products): full product line with specs
- [Pricing](https://www.acme.com/pricing): plans and volume discounts

## Docs
- [Getting started](https://www.acme.com/docs/start): setup in 10 minutes
- [API reference](https://www.acme.com/docs/api): REST endpoints and auth

## Optional
- [Company history](https://www.acme.com/about)

  3. Verify it is reachable and starts with the H1:

    curl -s https://yoursite.com/llms.txt | head -n 5

  4. Re-scan. Our validator checks the spec details: single H1 first, no H3+ headings in the intro, and at least one link bullet per H2 section.

FAQ

Will llms.txt improve my AI visibility?

Honest answer: probably not by itself, today. No major provider has confirmed it affects retrieval or citations. It is a curated site map that some AI tools read when present. Fix crawler blocks, rendering, and schema first — those have measurable effects — and treat llms.txt as polish.

Is llms.txt the same as robots.txt?

No, they are opposites. robots.txt controls access (who may crawl what) and is universally respected. llms.txt is guidance (here is my best content, summarized for LLMs) and is a voluntary convention with limited adoption.

What is llms-full.txt?

An optional companion file containing the full text of your key pages in Markdown, for tools that want content without crawling. It is even less standardized than llms.txt — only worth generating if you have documentation that AI coding tools consume.

Not sure if your site has this issue?

Run a free scan — we test this check (and 60+ others) against your live site and give you the exact paste-ready fix. No signup needed.