Validate llms.txt
Point it at a domain, or at a subpath file such as example.com/docs/llms.txt, and we fetch it the way a crawler would; or paste the text before you publish. You get two verdicts: spec validity, and a quality grade with the arithmetic behind it, every finding carrying its rule id and line.
Checked against llmstxt.org v2 (August 10, 2026) · All rules and deductions
What we check
Errors are format violations a crawler would trip on. Warnings are things that make the file less useful. Info is context that never changes the score.
Is this a valid llms.txt at all? Any of these failing makes the verdict “invalid”. Nothing here moves the quality score.
| Rule | What it checks | Basis | Effect |
|---|---|---|---|
| E1 | No file at the given path | v2: the file lives at /llms.txt or at a subpath such as /docs/llms.txt and covers the pages under it. | spec invalid; quality not assessable |
| E2 | Not served as text, or larger than 1 MB | v2 says the file must stay small enough to fit in context and is markdown; the text/* content type and the 1 MB bound are Studio limits. | spec invalid |
| E3 | No H1 title | v2: an H1 with the project or site name is the only required section. A leading byte-order mark is allowed. | spec invalid |
| E4 | Line inside an H2 section is not a file-list entry | v2: H2 sections hold file lists — list items with a required [name](url) link and optional notes after a colon. Prose belongs above the first H2. | spec invalid |
| E5 | HTML page served instead of the file | The response is a web page, not a markdown file — usually a catch-all route or a soft 404. | spec invalid; quality not assessable |
Studio's recommendations, scored 0–100 from a start of 100. None of these is required by the spec.
| Rule | What it checks | Basis | Effect |
|---|---|---|---|
| W1 | No blockquote summary (recommended) | v2 describes a blockquote summary after the H1 but does not require it. Studio quality check. | −10 each, max −10 |
| W2 | No H2 sections (recommended) | v2 allows zero or more H2 file-list sections. Without any, the file gives an agent nothing to follow. Studio quality check. | −20 each, max −20 |
| W3 | Dead link | v2: links should point to LLM-friendly content. A link that does not resolve points nowhere. Sample of 20 links. | −4 each, max −24 |
| W5 | More than 200 links | v2 sets no count, only that the file stays small enough to fit in context. 200 is a Studio threshold. | −5 each, max −5 |
| W6 | Duplicate URL | Not in the spec. Duplicate entries waste the context the file is meant to save. Studio quality check. | −2 each, max −10 |
Facts worth knowing. They change neither verdict.
| Rule | What it checks | Basis | Effect |
|---|---|---|---|
| I1 | robots.txt blocks documented AI crawler tokens | Which documented crawler tokens, including control-only tokens like Applebot-Extended, robots.txt disallows at /. Blocking is a choice, never scored. | none |
| I2 | llms-full.txt presence | llms-full.txt is not part of the llmstxt.org proposal; it is a convention some sites use for inlined content. Reported, never scored. | none |
| I3 | File age (Last-Modified) | How old the served file is, from the Last-Modified header when present. | none |
| I4 | rel="describedby" discovery | v2 recommends a <link rel="describedby"> element or an HTTP Link header pointing pages at the llms.txt that covers them. | none |
| I5 | Share of links resolving to markdown | v2: links should point to LLM-friendly content such as .md / .html.md versions. Share of sampled links that resolve as text/markdown or use a .md URL. | none |
| I6 | External links noted | v2 explicitly welcomes links to other sites; its own example links to GitHub. Listed for awareness, never scored. | none |
Link checks. URL mode and paste mode both sample up to 20 links, HEAD first then GET, same etiquette as the generator. A dead link is any 4xx/5xx or a connection failure; a redirect that lands is fine.
Off-domain links (W4). Links to another registrable domain are flagged, except well-known documentation hosts (readthedocs, GitHub Pages, GitBook, Mintlify, Notion, ReadMe).
robots.txt (I1). We list GPTBot, ClaudeBot, Google-Extended and PerplexityBot when they are blocked at the root. That is your call; the note is there so nobody expects a blocked crawler to read the file.
No file yet? Generate one.