Rubric changelog

Rubric versions, what changed, and the corpus impact of each change.

Rubric changelog

v2.1 — 2026-10-02

Rubric v2.1 makes Compliance & Trust opt-in and scores it rule by rule (T7.55; owner

decision 8; AItoolsplan.md "How playbooks change the P audits"). Spec version unchanged (llms.txt

v2). Fairness and gaming review: docs/research/rubric-v2.1-review-2026-10-02.md. Before and after:

corpus/derived/rescore-2026-10-02-v2.1.md.

Rule changes.

The gate is a declaration. playbook_matched is true when the covering llms.txt declares an

industry and a declared playbook can be scored. Syntax: one Industry: (or Industries:) line

before the first ## section (plain, > blockquote, list item or bold key), in ## Identity,

or in a section whose heading starts with "Compliance"; values separated by , ; | or +;

each a playbook slug or display name compared without case, spaces or punctuation

(Industry: Housing, General business). At most 3 (as in a playbook combination); further known

values are reported as over_limit, values we don't know as unknown; neither is scored. The

first such line counts; code fences are not read. Parsed as declared_industry (parse-core).

Only signed-off rules score. A declared playbook is scored when it is publishable and has a

signed-off (status: rule) blocking or high rule in a P1–P4 category with a matcher. Every

shipped playbook is still "to verify" today, so a declaration is recorded and reported ("its

playbook awaits reviewer sign-off") but the category stays Not applicable.

The domain list is retired. v2.0 matched 7 homebuilders and brokerages by domain

No domain opens the gate in v2.1.

Profile. A scored industry declaration (an industry-kind playbook) selects regulated.

The REGULATED domain regex is kept as the fallback for sites that declare no scored industry:

retiring it would have moved 22 undeclared sites to another profile, 17 of them down (up to

−1.9, wegmans.com and metlife.com D → F), which would make not declaring cost points.

P1–P4 measure coverage with a matcher per rule (tools/playbook-matchers.mjs, 325 rules, every

blocking/high P1–P4 rule of every playbook, signed off or not), severity-weighted (blocking 2, high

1); P3 (and P1, P2, P4) Not applicable where the scored playbooks define no rule of that kind; P4

half credit for a trigger without a route. The phrase proxies (compliance_language,

never_confirm) no longer score anything (the parse flags stay for research).

P5 reads a quoted or labelled not-advice disclaimer addressed to agents (was 0.7 for any

compliance phrase). P6 is measured (citation preference 0.5 + content licensing for AI use 0.5;

was a constant 0) and leaves the known-constants list (CONSTANT_SCORED is now R1, R2, R3).

AGENTS.md counts. A rule is found in the llms.txt or AGENTS.md; each file is matched on its

own and the breakdown keeps each file's coverage (scans.audits.compliance) with the scored

rules on P1–P4 (rules: short name, severity, credit), so the report is a gap list.

Draft re-score (T10.4). score-draft takes the playbooks: an llms.txt draft that declares or

drops an industry re-gates P1–P6 and the profile, recombined with the scan's AGENTS.md coverage;

an AGENTS.md draft re-checks P1–P6 against the scan's llms.txt coverage. A v2.0 scan is refused

(rubric_changed) as before, so no draft mixes two rubrics.

Fairness. Undeclared sites are scored exactly as v2.0 scored unmatched sites: the category is

removed and its weight redistributed, never zero. Declaring is opt-in to a stricter check: it can

lower a score until the file covers the playbook (simulated: the 7 former domain-list sites would

each score lower declared than undeclared today), which is the intended incentive (declare when

your file carries the rules), stated on the audit pages. The one site that loses points in this

rescore, www.khov.com (−0.9, stays C), lost credit it got from a domain list, not for not

declaring; it regains the category by declaring once the housing playbook is signed off.

Corpus impact (157 domains: the 156 of the published corpus plus www.khov.com, archived by a

live scan on 2026-07-29; corpus/derived/scores-2026-10-02-v2.0.ndjson → …-v2.1.ndjson, the

same parse, corpus/derived/parsed-2026-10-02.ndjson):

No file declares an industry, so P1–P6 apply to none (v2.0: 7).
Mean 57.63 → 58.09 (157); 57.51 → 57.98 over the 156. 6 up, 1 down, 150 unchanged. Grades

A/B/C/D/F 0/1/12/55/89 → 0/1/13/56/87: taylormorrison.com D → C (62.6 → 78.3),

tripointehomes.com F → D (50.9 → 63.6), compass.com F → D (49.4 → 61.7); opendoor.com +11.7,

lgihomes.com +10.8, tollbrothers.com +10.6 (all F). Each had a phrase-proxy compliance score of

0 at the regulated weight of 20. www.khov.com 75.9 → 75.0 (C): its proxy score was 79.5.

No profile, cap or non-P audit changes anywhere.

v2.0 — 2026-09-28

Rubric v2.0 judges files against llms.txt v2 (llmstxt.org, modified 2026-08-10). Every

scorecard and breakdown now records spec_version: "llms.txt v2 (2026-08-10)" next to

rubric_version. Why: llmsupdate926.md. Interfaces: tools/RUBRIC-V2-CONTRACT.md.

Owner decisions. Numbered 2.0 (a break, aligned with the spec), not 1.2. E5 is retired from

scoring; capture of llms-full.txt / llms-ctx*.txt continues for research. The agent test

(A10) is parked until the paid tier. Subpath scans are allowed in the free tier, with the same

quota.

Rule changes.

R1: any covering file is full credit. v2: "A file covers the URLs under its path, and where

more than one file applies, agents should use the most specific one." R1 is 1.0 for the most

specific covering file, root or path (was 0.5 at a subpath). Which file covers the scanned URL

is recorded in <date>.llms.meta.json and shown as R1's evidence and the scorecard's

evidence.covering_path; no meta file means the root file.

`no_file` cap: only when no file covers the URL at its path or any ancestor, up to the

root. The scan decides this before scoring (the engine only runs on a file).

S5 retired (## Optional present). v2 took away Optional's special meaning. Spec is now

S1–S4 at 25 each (were 20).

C4 is budget only. A file over 8k tokens with ## Optional no longer gets 0.85; it gets the

same budget score as any file its size.

C8 (new, Curation, weight 20, gate: file has links): share of the file's links that point to

LLM-friendly content, meaning a .md/.txt URL, or a page declaring a markdown alternate.

C1–C7 scaled by 0.8 (to 16/16/8/12/12/8/8) to keep their ratios.

E3: a page counts as having a markdown twin in either v2 form (page.html.md or page.md;

index.html.md or index.md), declared alternates first. That is a change to the sampler; the

engine still scores E3 as the sampled hit rate.

E5 retired (llms-full.txt). v2 removed context expansion.
E8 (new, Ecosystem, weight 9, gate: pages checked): pages declare

rel="alternate" type="text/markdown" (0.5) and rel="describedby" (0.5), from <head> or the

Link: header, averaged over the homepage and up to 3 sampled pages that answered 200

(<date>.pages.json). A link to a markdown version samples the page it mirrors

(/docs/method.md → /docs/method), so a file that links only to .md URLs still gets its

pages checked. UNVERIFIED when no pages were checked. E8 takes E5's 9, so E1–E7 keep

their weights. The §4.6 table now matches the registry (22/18/18/13/9/11/9).

WebMCP counts as a machine surface for A5/A6 (new parser flag mentions_webmcp: "WebMCP",

navigator.modelContext, document.modelContext, toolname=).

Self-blocking (R5, E1, `self_blocking` cap) follows RFC 9309. Before, a reader counted as

blocked only on a literal Disallow: / under the last User-agent line. Now it is blocked when

tools/robots.mjs isAllowed says it may not fetch the covering llms.txt. So consecutive

User-agent lines share their rules, an Allow: /llms.txt carve-out is honoured (not blocked),

Disallow: /llms.txt or a Disallow over the file's subpath is caught, and a reader with no

group of its own falls back to User-agent: *. User-agent: * + Disallow: / counts as

self-blocking: the file is unreadable to every reader, which is exactly what R5 ("AI crawlers

permitted to fetch it") and the cap describe. The reader set and the named count are

unchanged.

Still constant (the §5 ratchet in tools/test/score.test.mjs): R1, R2, R3, A4, A6, A9, P6.

R1 stays constant by design, since every covering file now earns 1.0. C8 and E8 launch measured,

and the ratchet asserts that both apply and vary.

Corpus impact (156 domains; the same capture as v1.1, which reproduces exactly from the v1.1

engine; corpus/derived/scores-2026-09-28-v2.0.ndjson against …-v1.1.ndjson):

Mean total 60.42 → 57.96 (−2.46): 38 up, 101 down, 17 unchanged. Grades: 6 up, 31 down

(A/B/C/D/F 0/3/17/67/69 → 0/1/13/55/87). Most of the downward moves are D → F near 60.

C8 accounts for the drop (mean −2.49 on its own). It applies to 145 files, with a mean

of 0.33: 75 have no .md/.txt links (0), 47 score ≥ 0.5 and 19 score 1.0. Docs sites that

link markdown gain (mintlify.com and datadoghq.com +4.5, platform.claude.com +4.3). Brand sites

linking HTML pages lose up to ~6 (siemens.com −6.3, target.com −6.2, kaggle.com −6.2).

retool.com and zapier.com fall from B to C.

S5 and C4. S5 applied to 38 files and credited 8, so 30 long files without ## Optional

gain Spec points (Spec mean +2.75). C4 changes on the 8 files that had the escape: 3 up

(bun.sh, docs.perplexity.ai, together.ai: the budget alone scores above 0.85) and 5 down

(posthog.com to 0.3, docs.stripe.com to 0.55). Net effect on the mean: +0.27.

E5. Applied to 38 files, credited 13. Net effect on the mean: +0.08.
Self-blocking. 3 → 4 capped. The new one is khov-dev (www-dev.khov.com, the staging host):

User-agent: * / Disallow: / makes its file unreadable to every reader, so it falls from 72.8

to 25. The other three stay capped, and no site loses its cap: R5 and E1 change on khov-dev

only.

R1, E8, WebMCP: no corpus effect yet. Every archived file came from /llms.txt and none has

a llms.meta.json. No pages.json exists, so E8 is UNVERIFIED on all 156. The one file that

mentions WebMCP (zapier.com) was already credited for MCP.

Rank agreement with the Edition 1 judgments (Kendall tau, n = 55, method of

tools/render-rankings.mjs): 0.374 → 0.335. Without khov-dev it is 0.369 → 0.364. Nearly all of

the fall is khov-dev, which Edition 1 judged on content (77) while v2.0 caps it for being

unreadable. Retiring S5 and the C4 escape costs 0.016 of the rest, because Edition 1's judges

worked from v1 and praised ## Optional. C8 adds 0.002 (a C8 weight of 10 would score 0.325).

Measurement fixes (2026-09-29). Two engine gaps found while writing the audit FAQs

(docs-accuracy.md); no weights changed, and rubric_version stays 2.0 (for A7.3–A7.5 too, by

owner decision 11.6). The only gate changes are E3's md_twins_sampled, which now receives a

sample, E4's jsonld_checked and E6's canonical_checked, which are now computed from the page

capture, and A6's has_actions and A4's multi_journey, which now read an action surface and

evidence of multi-step work (all below).

A7 accepts `## FAQs`. The heading test was /\bfaq\b|frequently asked/i, so a section

named "FAQs" scored 0.2; it is now /\bfaqs?\b|frequently asked/i.

E1/R5 naming reads a versioned `User-agent`. User-agent: GPTBot/1.0 now names GPTBot:

the /… suffix is dropped before matching, as the RFC 9309 blocking rule (tools/robots.mjs)

already did, so naming and blocking read the same line the same way. The scan progress line

(worker/src/progress.mjs) follows suit and, like the engine, ignores a trailing # comment.

Live scans fetch `/CLAUDE.md` and `/SKILL.md`. Live capture fetched only /AGENTS.md, so a

site with only a CLAUDE.md scored 0 on E7 on a scan but 0.6 in the research corpus. It now

fetches all three, as corpus/capture.sh does, with the same acceptance rule (HTML, non-200 and

under 20 bytes rejected) and archives them under the same names. The scan progress line names

CLAUDE.md / SKILL.md only when found.

Corpus impact: none (node tools/audit-stats.mjs, 156 domains: every figure unchanged).

No archived file names a section "FAQs" (A7 full marks stay 5 of 156, 3.2%), and no archived

robots.txt puts a version on an AI crawler's User-agent (E1 full marks stay 30 of 148,

20.3%).

Docs follow the engine. The per-audit docs and the §3.3 and §4 tables now state the

engine's actual formula and gate for every audit, marking the rubric's intended measure as

Planned where the engine uses a proxy or a constant.

A URL without a trailing slash can be a directory. Resolution treated the last path

segment of https://x.com/docs as a page and never tried /docs/llms.txt. A last segment with

no file extension is now also tried as a directory, most specific first (/docs/guide tries

/docs/guide/llms.txt, /docs/llms.txt, /llms.txt); one with an extension (/docs/page.html)

stays a page. The cap of five candidates, root always last, is unchanged; a scan of an

extension-less deep URL makes at most one more request.

401 is turned away, not Unverified. A 401 on the llms.txt fetch means authentication is

required: the file is not publicly retrievable, which is a fact about access, like a 403, not a

measurement failure. The scan outcome is now BLOCKED ("turned away at the door", copy now

covers a login) instead of UNVERIFIED (worker/src/score-bridge.mjs classifyOutcome).

Neither outcome is scored, so no total changes; only the report the site sees does.

A robots.txt 4xx means no rules (RFC 9309 §2.3.1.3). A robots.txt that answers 4xx is

"unavailable", and crawlers may access everything. R5 and E1 were UNVERIFIED for it; they now

score as a robots.txt with no rules: nothing blocked, no AI crawler named, so 0.5 each and no

self_blocking cap. The engine only does this when the capture recorded the 4xx: the scan

(worker/src/score-bridge.mjs) and corpus/capture.sh write <date>.robots.meta.json

({ url, status }), which robotsPolicy reads. No record, a 5xx, no response, or a 429 (a

rate limit on our request; Google also treats it as a server error) stays UNVERIFIED. When an

archive holds both, the newer day wins, and on the same day the file we read does.

A short robots.txt is archived. The capture rule "a 200 under 20 bytes is empty" threw away

complete robots.txt files such as User-agent:* / Allow: (19 bytes). robots.txt now only

needs a 200 and a non-HTML body (worker/src/accept.mjs, corpus/capture.sh); an empty 200

robots.txt is also a file with no rules and scores 0.5 on R5 and E1.

Markdown versions (E3) are measured, live and in the corpus (A5.1). E3 was UNVERIFIED on

every scan because nothing passed the twin sample to the scorer (mdTwins: null). The sampler

(tools/sample-md-twins.mjs sampleTwins) now runs on every live scan (worker/src/capture.mjs

fetchMdTwins, after the pages capture so declared alternates are tried first; LLMSRank-User,

like the rest of the scan) and writes <date>.md-twins.json, which the engine reads for E3

(tools/score.mjs mdTwinSample). Cost per scan: 3 pages in parallel, at most 4 candidate URLs

each (HEAD, a ranged GET only where HEAD is refused), 5 s per request, 10 s for the whole sample;

what the budget cuts off is unverified, never a miss. Scoring is found ÷ pages answered

definitively (200, 404, 410); a 403, 429, timeout or robots.txt block leaves the page out, and

zero definitive pages is UNVERIFIED. A file with no URLs at all is now NOT_APPLICABLE. The scan

progress line reads "markdown versions · 2 of 3 found".

A linked page served as markdown is its own markdown version. A link such as

https://www.pinecone.io/product/md/ that answers text/markdown was scored a miss, because a

directory URL only tried index.html.md and index.md. The page capture (<date>.pages.json)

now records the content type of a non-HTML page, and a page that answered text/markdown or

text/plain counts itself first. No extra requests.

Corpus impact of E3 (node tools/audit-stats.mjs, 156 domains; twins sampled 2026-09-29 by

LLMSRankBot behind robots.txt, 6 sites at a time, corpus/files/<domain>/2026-09-29.md-twins.json

and corpus/derived/md-twins-2026-09-29.ndjson; declared alternates read from the page heads,

not archived as pages.json, so E8 stays as it was): E3 applies to 123 sites, is UNVERIFIED on 31

(17 with no same-site page links the parser reads, 14 where every check was blocked, rate-limited

or timed out) and NOT_APPLICABLE on 2. 42 of the 123 (34.1%) score 1.0, 70 (56.9%) score 0, mean

0.388. 22 hits came through a declared alternate or a page served as markdown. Totals: mean 58.0

→ 57.5, median 59.4 → 58.5; 44 up, 72 down, 40 unchanged; grades 10 up, 14 down (D 55 → 51, F 87

→ 91). The largest moves are developer.civitai.com +13.0 (F → D) and jasper.ai −4.3.

Corpus impact of these three: none (node tools/audit-stats.mjs, 156 domains: every figure

unchanged). No corpus llms.txt fetch recorded a 401 (<date>.meta.json http_status: 153

× 200, 4 × 403, 1 malformed), and the 2026-07-19 capture recorded no status for a robots.txt it didn't archive, so the

8 domains with no archived robots.txt stay UNVERIFIED on R5 and E1 until they are recaptured.

JSON-LD agreement (E4) is measured (A5.2). E4 was UNVERIFIED on every scan because page

markup wasn't captured. The page capture that E8 already runs (tools/capture-pages.mjs, live

and corpus) now also records each page's identity JSON-LD (tools/jsonld.mjs extractJsonLd):

only @type, name, alternateName, legalName, url and sameAs, at most 8 entities per

page, stored in pages.json (jsonld, format in tools/RUBRIC-V2-CONTRACT.md). No extra

requests. The read still stops at </head> when the head carries identity JSON-LD; otherwise it

continues into the body, to </html>, within the same 256 KB cap, because many sites emit

JSON-LD there (43 of the 103 corpus sites with identity JSON-LD have it only in the body,

stripe.com and vercel.com among them): reading the head alone would have scored them 0.

<link> relations are still read from the head only. A block that doesn't parse counts as

absent and never fails the capture. Rule: 0.4 identity present + 0.4 a name agrees with the H1

+ 0.2 a url on the file's site (§4.6, with the reasons for the thresholds). The gate

(jsonld_checked) needs a page that answered 200 and was read for JSON-LD, and, for a 0, the

homepage read to the end. A pages.json captured before this change has no jsonld, so E4

stays UNVERIFIED for it while E8 still scores.

Corpus impact of E4 and E8 (node tools/audit-stats.mjs, 156 domains; pages captured

2026-09-29 by LLMSRankBot behind robots.txt, 6 sites at a time, one

corpus/files/<domain>/2026-09-29.pages.json per domain with an archived llms.txt, 157 written).

The corpus had no pages.json until now, so E8 is measured across the corpus for the first

time: it applies to 142 sites and is UNVERIFIED on 14 (every page answered 403 or 429, robots.txt

disallowed them, or nothing is archived). None earns full marks, 97 (68.3%) score 0 and 27 score

0.5; mean 0.129. 40 sites declare a text/markdown alternate on at least one page and 6 declare

describedby. The same captures feed C8's declared alternates: 12 sites move, C8 mean 0.329 →

0.338, full marks 19 → 20. E4 applies to 123 sites and is UNVERIFIED on 33 (the same 14, plus

19 whose homepage was cut at the 256 KB cap or didn't load and no page showed identity). 94 of

the 123 (76.4%) score 1.0, 20 (16.3%) score 0 and 9 are partial (5 × 0.8, 3 × 0.6, 1 × 0.4);

mean 0.815. Totals: E8 (with C8) alone takes the mean from 57.50 to 56.78 (125 sites down), E4

then adds 0.70 (97 up); together mean 57.50 → 57.48 (57.5 as published), median 58.5 → 58.3; 77

up, 53 down, 26 unchanged; grades 5 up and 5 down, so the counts are unchanged (B 1, C 13, D 51,

F 91). The largest moves are ollama.com +4.3, n8n.io +3.6 (F → D) and developer.civitai.com −5.6

(D → F).

Canonical agreement (E6) is measured (A5.3). E6 was UNVERIFIED on every scan. The page

capture now also records each page's canonical (canonical: the head <link rel="canonical">,

else the Link: header, resolved to an absolute URL; tools/links.mjs canonicalOf), and

the engine compares it with the URL the llms.txt links (tools/canonical.mjs): 1 equal, 0.5 the

same page in another form, 0 a canonical elsewhere; pages with no canonical are left out (rule

and reasons in §4.6). No extra requests. No corpus recapture was needed: the 2026-09-29

pages.json already keeps every head <link> element (and the raw Link: header), so

canonicalOf derives the canonical from those, resolved against the page URL, and pages.json

stays write-once; only new captures (and live scans) store canonical. The one loss: a head with

more than 100 <link> elements is cut at 100 in the capture, so a canonical after the hundredth

is missed and the page is left out (18 of 502 corpus pages reach the cap).

Corpus impact of E6 (node tools/audit-stats.mjs, 156 domains, same 2026-09-29 captures).

E6 applies to 111 sites and is UNVERIFIED on 45: 14 where no page loaded (the sites E8 also

leaves out) and 31 where a page loaded but none of the sampled linked pages declared a canonical

(a homepage-only capture, or no rel=canonical). Of the 330 linked pages compared, 308 are equal, 1

is the same page in another form and 21 point elsewhere (dell.com's links keep tracking

parameters and pluralise a path; docs.windsurf.com links pages whose canonical is on docs.devin.ai;

suno.com/explore canonicalises to the homepage). 96 sites (86.5%) score 1.0, 3 (2.7%) score 0, 12

are partial; mean 0.928. Total movement: mean 57.48 → 57.94 (57.5 → 57.9 as published), median

58.3 → 59.1; 98 sites up, 8 down, 50 unchanged; grades 4 F → D and 2 D → C (F 91 → 87, D 51 → 53,

C 13 → 15; B 1). The largest moves are cox.com and americanexpress.com +1.5; the falls are

under 1 point and are sites whose linked pages all point elsewhere (decagon.ai and

docs.windsurf.com, −0.8; lmstudio.ai −0.7).

Entity identity (A8) is measured from the file (A5.4). A8 scored 0.6 with a version or

date stamp and 0.2 without, so identity itself was never read and nothing could score above 0.6.

The parser (tools/parse-core.mjs identitySignals, synced to the browser copy) now sets four

flags from the file's prose, never its link lines or code: identity_statement,

alternate_names, identifiers and glossary. Rule: 0.4 an identity statement + 0.3

alternate names or identifiers + 0.3 a glossary; 0 when the file states none (rule and reasons

in §4.4). The version stamp left A8 (it stays in C7), the jargon gate was dropped (no

deterministic test for jargon), and page JSON-LD does not corroborate (E4 scores it, and in the

corpus the legal names it declares appear in files only incidentally). Deterministic, no model.

The corpus parse (corpus/derived/parsed-2026-07-19.ndjson) gains the four flags, computed by

the current parser from the same archived files; nothing else in it changes, and a fresh

parse-llms --all differs from the old output only in these flags.

Precision of the A8 signals (every corpus match read by hand, 156 files): identity statement

20 files, all naming the file's own entity (3 name a subsidiary or project entity of it: K.

Hovnanian American Mortgage, LLC; Toll Northeast V Corp.; UBS Limited); alternate names 2 (both

former names: Cencora, HII); identifiers 11 (8 tickers, an NMLS number, "the official website

of", a labelled primary domain), no false positive; glossary 11 (4 with a glossary heading, 7 through a linked

glossary page only), no false positive. Near misses excluded by the rule: "also known as RAG" (a term, not the

entity), Vercel's ### aliases, DeepL's "Create a glossary" endpoint, Synthesia's

translation-glossary feature, "Key terms: Agile, Scrum…" keyword lists, link descriptions

naming a company ("Rockwell Automation, Inc."). Known misses, by design: identity stated only in

a link title or description, and glossary pages whose URL isn't /glossary (insurance-glossary).

Corpus impact of A8 (node tools/audit-stats.mjs, 156 domains). A8 still applies to all

156. Before: 16 × 0.6 (stamp), 140 × 0.2, mean 0.241. After: 35 sites (22.4%) score above 0 — 7 ×

0.7 (identity plus identifiers or former names: albemarle.com, cencora.com, expedia.com,

hii.com, t-mobile.com, tollbrothers.com, khov-dev), 1 × 0.6 (cognizant.com: ticker and glossary),

13 × 0.4 (identity only), 14 × 0.3 (a glossary or identifiers only); 121 (77.6%) score 0; none

earns full marks (no file pairs an identity statement with a glossary); mean 0.096. Totals: mean

57.94 → 57.53 (57.9 → 57.5 as published), median 59.1 → 58.4; 29 up, 118 down, 9 unchanged (all

capped); grades 6 down (developers.deepl.com C → D; docs.unstructured.io, docs.leonardo.ai,

dxc.com, wegmans.com and paypal.com D → F): B 1, C 15 → 14, D 53 → 49, F 87 → 92. The largest

moves are hii.com +1.8, expedia.com, albemarle.com and t-mobile.com +1.7, and the falls of sites

whose stamp earned 0.6 while Agent Readiness had few applicable audits (harley-davidson.com −4.1,

americanexpress.com −3.8).

API error handling (A9) is measured from the file (A7.3). A9 was a constant 0.2 (min(0.6,

judged A5) with an LLM audit) wherever an API was documented. The parser now sets api_errors,

api_limits, api_auth and api_recovery, and A9 scores the parts found ÷ 4 (rule and reasons

in §4.4). The gate is unchanged (has_api) and the judged overlay is gone (owner decision 11.3).

The report line names each part: "API errors: documented; rate limits: documented; auth:

documented; recovery: not found".

Action endpoints (A6) are measured, and A6 has its own gate (A7.4). A6 was a constant 0.7

gated on the API signal, so a dentist's booking link never counted and an API-only docs site

always did. The gate is now an action surface (has_actions: api_write_operation,

mcp_endpoint, mentions_webmcp, action_link, action_prose), and the score is 0.5 for the

action plus 0.5 for guidance on when to offer it (offer_guidance). The report line lists the

surfaces found and whether the file says when to offer them.

Journey flows (A4) are measured, and A4's gate asks for multi-step evidence (A7.5). A4 was

0.8 wherever intent-routing wording appeared (0.8 × judged A3 with an LLM audit). The parser

now counts flows (journey_flows) and whether one links its steps (journey_linked_steps) or

states a prerequisite or handoff (journey_prereq_handoff); A4 scores 0.4/0.6 + 0.2 + 0.2. The

gate is an agent-callable action or a flow (owner decision 11.2). The judged overlay is gone

(11.3). Deterministic, no model, for all three.

Still constant (the §5 ratchet in tools/test/score.test.mjs, mirrored by the app's

CONSTANT_SCORED): R1, R2, R3, P6. A4, A6 and A9 left the list, and the ratchet now also fails

if any of the three stops varying across the corpus files.

Corpus parse. corpus/derived/parsed-2026-07-19.ndjson gains the 12 new flags, computed by the

current parser from the same archived files, plus mentions_webmcp, which the parser has set

since v2.0 but this file predated: A6's and A4's gates read it, and zapier.com's only

agent-callable surface is its WebMCP manifest. Every other field is unchanged (each record,

with these flags removed, is byte-identical to the previous line). No other audit reads

mentions_webmcp differently: zapier.com, the one file that mentions WebMCP, already had MCP.

Corpus impact of A9, A6 and A4 (node tools/audit-stats.mjs, 156 domains, with the A7.10

fixes below; at A7.5 every per-site total equalled the frozen estimate in

docs/research/a4-a6-a9-patterns-2026-09-29.md). A9 still applies to 74: 36 × 0, 16 ×

0.25, 13 × 0.5, 9 × 0.75, none at 1.0; mean 0.2 → 0.233. Parts: auth 26, errors 22, limits 12,

recovery 9. A6 applies to 83 (was 74; 88 before the A7.10 fixes): 45 stay, 38 newly apply,

29 leave (API docs with no action surface, such as developers.cloudflare.com and github.com, and

the 5 whose only surface is a docs-search or read-only MCP server). Surfaces: 31 agent-callable

(23 API write operations, 8 declared MCP endpoints that take actions, 1 WebMCP; 6 more files

declare only a docs-search or read-only MCP server) and 57 user-facing (49 action links, 11

prose actions; 34 files have contact links only). 74 × 0.5 and 9 × 1.0 (aflac.com, attio.com,

docs.stripe.com, expedia.com, khov-dev, mccormick.com, ollama.com, trivago.com,

wyndhamhotels.com); mean 0.7 → 0.554. A4 applies to 35 (was 1; 40 before the A7.10 fixes):

before the Quickstart partial credit (owner decision 2026-10-01, below), 28 × 0, 5 × 0.4

(attio.com, carvana.com, firecrawl.dev, lmstudio.ai, trychroma.com), 1 × 0.8 (mccormick.com),

1 × 1.0 (khov-dev); mean 0.8 → 0.109. With it, 17 of the 28 score 0.2 and 11 stay at 0; A4's

mean is 0.206, the mean total 57.51 (57.5 as published) and the median 58.8, grades unchanged.

The figures that follow are before it. On its own (frozen estimate, before A7.10), A9 moves

the mean total 57.53 → 57.55, A6 → 57.60 and A4 → 57.34. Together: mean 57.53 → 57.45 (57.5 as

published; 57.44 at A7.5), median 58.4 → 58.55 (58.6 as published then); 40 up, 65

down, 51 unchanged; grades 8 up and 8 down (B 1, C 14 → 11, D 49 → 55, F 92 → 89). Up to D:

dxc.com, westerndigital.com, wegmans.com, mccormick.com, paypal.com, albemarle.com, lumen.com,

paychex.com. Down: apify.com, clerk.com and docs.poolside.ai (C → D); elevenlabs.io,

posthog.com, pinecone.io, browserbase.com and ziprecruiter.com (D → F). The largest falls are

API sites with no flows and no offer guidance (developers.cloudflare.com and

developers.llamaindex.ai −2.7, prisma.io −2.4, augmentcode.com −2.3); the largest gains are sites

with action links, offer guidance or a linked flow (mccormick.com +2.8; sentry.io, mistral.ai,

wyndhamhotels.com and aflac.com +2.6; expedia.com +2.4).

A7.10 fixes (gaming and fairness review, AIplan.md item 11). Five rule changes, each with

the review's crafted gaming file as a test: (1) a docs-search or read-only MCP server (its line

mentions /_mcp/server, an mcp-docs. host, "docs MCP" or documentation, or its H2 section

says "read-only") still counts as declared and for A5, but no longer opens A6 or A4

(mcp_docs_only, a new corpus parse flag; every other field of each record is unchanged); (2) an

A4 flow needs more than half its steps with ≥ 2 words or a link or operation, and an arrow chain

needs its intro word and a colon before the first arrow; (3) an A6 guidance heading needs a body

and the same verb test as prose; (4) A9 recovery prose needs an idempotency key, not the bare

word; (5) an A9 heading's body needs a link, URL, code or a valued prose line. Fixes 2–5 change

no corpus figure. Fix 1: A4 applies 40 → 35 (mean 0.095 → 0.109), A6 88 → 83 (0.551 → 0.554),

mean total 57.44 → 57.45, median unchanged; dev.hume.ai 68.9 → 70.2 (D → C), you.com 74.4 →

75.5, assemblyai.com and docs.docker.com −0.1, ziprecruiter.com unchanged (its A6 0.5 and A4 0

cancel); grades C 10 → 11, D 56 → 55. Fern's generated "connect to the MCP server at

…/_mcp/server" line, which dev.hume.ai, developers.deepgram.com and you.com share, had cost

dev.hume.ai its C. llmsrank.dev's own file scores the same under all five (A4 0, A6 1.0, A9 0.5).

Owner decision 2026-10-01: Quickstart partial credit. The A7.10 review measured, and didn't

recommend, 0.2 pointer credit for a linked Quickstart page when no flow is written (AIplan.md

item 11, decision 11.8); the owner chose it. When A4 applies and journey_flows is 0, a link

line whose whole normalised title is "quickstart", "quick start", "getting started" or "get

started" (optionally + "guide" or "tutorial"; ≤ 120 characters; a page, not an in-file anchor)

scores 0.2 (journey_quickstart_link, a new corpus parse flag; every other field of each record

is unchanged). The rule is tighter than the review's (any list-item title containing the words,

23 files): whole titles only, so "Getting started with feature flags", "W&B quickstart" and

"API quickstart" don't count. It never adds to a flow's score and never opens the gate. Report

line: "no numbered or arrow flow with a journey intro; a quickstart or getting-started page is

linked (partial credit)". Corpus: A4 applies to 35, unchanged; 17 move 0 → 0.2 (11 × 0, 17 ×

0.2, 5 × 0.4, 1 × 0.8, 1 × 1.0); A4 mean 0.109 → 0.206 (zero rate 0.80 → 0.31). Mean total

57.45 → 57.51, median 58.55 → 58.8; no grade changes (B 1, C 11, D 55, F 89). 14 totals move,

+0.5 to +0.8: mintlify.com 72.4 → 73.0, developers.deepl.com 68.1 → 68.8, developers.deepgram.com

67.9 → 68.5, dev.writer.com 66.9 → 67.4, docs.perplexity.ai 66.4 → 67.0, docs.cartesia.ai 61.4 →

62.0, inference-docs.cerebras.ai 61.3 → 62.0, docs.fireworks.ai 61.3 → 61.9, weaviate.io 60.2 →

60.9, docs.unstructured.io 58.2 → 59.0, docs.leonardo.ai 58.2 → 58.8, docs.langchain.com 57.3 →

58.1, vercel.com 54.5 → 55.1, platform.claude.com 49.1 → 49.7. algolia.com, twilio.com and

together.ai gain A4 0.2 but stay at their caps (30, 30, 25). llmsrank.dev's own file has two

flows and scores the same (A4 1.0).

v1.1 — 2026-09-28

Measurement fixes from updatesuggestions.md; no weights changed.

C6 is measured. It was a constant 0.6. It now uses the archived sitemap per §4.3 and has

its own gate (sitemap_locs), so a sitemap we can't read completely makes C6 UNVERIFIED

without switching off E2. Sitemaps are captured by tools/capture-sitemaps.mjs.

C6 doesn't apply below 50 sitemap pages. On a small site a complete file and a sitemap

mirror look the same (lumalabs.ai: 23 pages, 18 linked), so C6 is NOT_APPLICABLE there.

E2 is measured. It was a constant 0.9 for "present". Now: 0.4 for a readable sitemap

+ 0.3 × share of pages with a lastmod + 0.3 × share of those that are plausible (a valid

W3C date, not after the capture day, and not one timestamp on ≥ 90% of a 10+ page sitemap,

which is a build time rather than an edit date). Index children that were archived count

even when the index is incomplete. No readable entries = UNVERIFIED.

E7 needs a real file. AGENTS.md or CLAUDE.md under 200 bytes no longer earns credit.
Corpus impact (156 domains, sitemaps captured 2026-09-28): C6 applies to 98 sites, is

NOT_APPLICABLE on 2 and UNVERIFIED on 56; the sites it scores down are mostly docs sites whose

llms.txt lists ~80–98% of their sitemap. E2 applies to 130 sites, mean 0.84: 61 score 1.0 (60 as first published; the same capture

and engine reproduce 61, corrected 2026-09-29) and

24 with no lastmod score 0.4. Against v1.0 the mean moves +2.26 (115 up, 6 down; 24 grades

up, 1 down), and rank agreement with the Edition 1 judgments goes 0.309 → 0.374.

Grades follow the stored total. The engine stored totals to one decimal but graded the unrounded value, so an unrounded 59.96 was stored as 60.0 with an F. It now grades the stored total. Corpus impact: 2 domains (dell.com, developers.llamaindex.ai) move from F to D; no score changes.
AI crawler list. R5/E1 recognise 23 AI crawler and control tokens (NAMED_TOKENS in tools/ai-crawlers.mjs), up from 13: 21 from the vendors' own documentation, each with its source, plus Bytespider and cohere-ai, kept because they are common in robots.txt files although no vendor page documents them. R5/E1 also ignore trailing # comments on User-agent lines (RFC 9309). The self-blocking set is unchanged. Corpus impact: none (every site naming a newer token already named an older one).
Still constant (tracked in tools/test/score.test.mjs, which fails if the list grows):

R1, R2, R3, A4, A6, A9, P6.

`robots_archived` gate fixed (D0.5). A missing or unreadable robots.txt made R5 and E1

NOT_APPLICABLE; §3.1 says a measurement we couldn't make is UNVERIFIED. Gate is now

robots.state === OK ? true : UV. Corpus impact (156 domains, same capture): 8 domains

(algolia.com, developer.civitai.com, docs.synthesia.io, gm.com, kaggle.com, ollama.com,

qdrant.tech, t-mobile.com — the ones with no archived robots.txt) move R5 and E1 from

NOT_APPLICABLE to UNVERIFIED; total is unchanged for all of them (mean Δ = 0), since both

states drop the audit and redistribute its weight the same way — only the NA/UV counts shown

in the report change.