Skip to content
LLMSRank
Docs menu

Curation, not a sitemap

Curation, not a sitemap mirror

In short: Link the pages that answer questions, not your whole sitemap: keep the file to 40% or less of the pages your sitemap lists.

  • Category: Curation Quality · weight 8 of 100 within the category
  • Curation Quality is worth 25 in generic and commerce, 20 in regulated and technical (points of 100, by scoring profile)
  • Audit ID C6 (as shown on reports) · rubric v2.1

What it checks

Whether the file curates your site rather than mirroring your sitemap.

Why it matters

An llms.txt that lists every page in your sitemap gives an agent no guidance about what matters. The value of the file is the choosing.

When it applies

Applies when a sitemap.xml exists and we could read all of it, and it lists at least 50 pages. Under 50 pages it’s Not applicable: on a small site a complete file and a sitemap mirror look the same. With no sitemap to compare against, or one we couldn’t read completely, it’s Unverified.

An audit that doesn't apply, or that we couldn't verify, is removed from your score and its weight shared among the Curation Quality audits that do apply. It is never scored as a zero.

How it's scored

1.0 at ≤ 40% overlap with sitemap.xml · linear to 0 at 100%. Overlap = share of the sitemap’s pages the file links to, compared by host (case and www. ignored) and path (trailing slash and .md ignored). UNVERIFIED when no sitemap is archived, it’s empty or truncated, or it’s an index whose children weren’t all archived (v1.1).

How to fix

  1. Link the pages that answer questions, not every page you have.
  2. Drop auto-generated lists (every blog post, every product variant) and link their index page instead.
  3. If you generate the file from your sitemap, filter it by hand afterwards.

Example

## Blog

- [All posts](https://acme.com/blog.md): Index of every post, newest first
- [How we price cross-border payments](https://acme.com/blog/cross-border-pricing.md): The post customers cite most

Frequently asked questions

How does LLMSRank decide my llms.txt mirrors my sitemap?
It measures the share of your sitemap’s pages that the file links to. Up to 40% scores full marks; above that the score falls linearly to 0 at 100%, so linking 70% of the sitemap scores 0.5. URLs are compared by host (case and www. ignored) and path (scheme, trailing slash, fragment and a .md suffix ignored), so linking the markdown version of every page still counts as mirroring it.

Sources: LLMSRank — Curation, not a sitemap audit

Why does this check say Not applicable or Unverified for my site?
It is Not applicable when your sitemap lists fewer than 50 pages, because on a small site a complete llms.txt and a sitemap mirror look the same. It is Unverified when we have no sitemap to compare against or couldn’t read all of it: empty, truncated, or a sitemap index whose child sitemaps weren’t all captured. Either way it is removed from your score, never counted as zero.

Sources: LLMSRank — Curation, not a sitemap audit · sitemaps.org — Sitemaps XML format

Isn’t a sitemap enough for AI agents?
The proposal says no: a sitemap “isn’t a substitute for llms.txt” because it often won’t list the LLM-readable versions of pages, leaves out helpful external URLs, and covers more than fits in a context window. A single sitemap file may list up to 50,000 URLs; an llms.txt is the curated overview of the few that answer questions.

Sources: llmstxt.org — The /llms.txt file, v2 · sitemaps.org — Sitemaps XML format · LLMSRank — llms.txt, robots.txt and sitemap.xml

I generate llms.txt from my sitemap. How do I pass?
Filter after generating: drop auto-generated runs such as every post, tag or product variant and link their index page instead, until the file links 40% or fewer of the sitemap’s pages. Links to other hosts, such as a GitHub repository, don’t count as overlap: they only match pages your sitemap lists, and sitemaps.org says a sitemap’s URLs come from a single host. The consistency checker compares your llms.txt links with your sitemap and gives the exact line to fix.

Sources: LLMSRank — Curation, not a sitemap audit · sitemaps.org — Sitemaps XML format

How many real llms.txt files are sitemap mirrors?
In our research, 99 of the 187 sites in our corpus (captured 2026-10-10, scored with rubric v2.1) had a sitemap of at least 50 pages that we could read in full; 87 of them (87.9%) earned full marks and the mean was 0.896. The check was Unverified for 86 sites and Not applicable for 2 — more Unverified than in July, because the corpus has grown faster than our sitemap captures have. netlify.com, aflac.com, you.com, cloudflare.com and www.khov.com are examples.

Sources: LLMSRank — Method · LLMSRank — Curation, not a sitemap audit