# llms.txt: tools and guidance This section explains llms.txt, a proposed markdown file that gives AI agents a short, curated guide to a website, and how to publish one that works. The guides describe the proposal as its own text at llmstxt.org does (v2, August 2026) and quote it where precision matters. Where they describe how LLMSRank scores a file, they say so: our rubric is our opinion, not the standard. Updated 2026-09-29. This is the markdown version of https://llmsrank.dev/learn. ## Check your site [Scan a domain](https://llmsrank.dev/) and LLMSRank fetches its robots.txt, llms.txt, llms-full.txt, AGENTS.md and sitemap.xml, then returns a graded report: what works, what stops agents reading the file, and the first fix to make. The scan is free, up to five a day. Chrome’s Lighthouse also has an llms.txt audit, in its experimental Agentic Browsing category. It checks that the file can be fetched: it fails on a server error and marks a missing file (404) as not applicable. It does not look at what the file says. LLMSRank grades the contents. ## Learn - [About llms.txt](https://llmsrank.dev/learn/llms-txt): what the file is, where it lives, its format, how agents use it, and what it is not. - [Writing an llms.txt file](https://llmsrank.dev/learn/writing-llms-txt): a step-by-step with two complete example files, and the mistakes we see most. - [Markdown page versions and link relations](https://llmsrank.dev/learn/markdown-pages): the `.md` URL convention and how pages point agents to their markdown version and their llms.txt. - [llms.txt, robots.txt and sitemap.xml](https://llmsrank.dev/learn/llms-txt-robots-txt-sitemap): what each file does, who reads it, and why none replaces the others. - [Does ChatGPT, Claude, Perplexity or Google read llms.txt?](https://llmsrank.dev/learn/does-ai-read-llms-txt): what each AI vendor has documented, with dates, and what independent log studies found. - [llms.txt vs AGENTS.md vs robots.txt](https://llmsrank.dev/learn/llms-txt-vs-agents-md): three files, three readers, compared. - [How to check your llms.txt](https://llmsrank.dev/learn/check-llms-txt): the free scan, a manual checklist mapped to our audits, and checks an AI agent can run. - [llms.txt generators: what generated files miss](https://llmsrank.dev/learn/llms-txt-generators): what generators do, and the editorial checks they leave to you, measured against the llms.txt Index. - [llms.txt release notes](https://llmsrank.dev/learn/release-notes): what changed in the proposal, version by version, newest first. ## Reference How LLMSRank turns a file into a score. This is our rubric, not part of the llms.txt proposal. - [Scoring method](https://llmsrank.dev/docs/method): the six categories, profile weights, grade bands, score caps, and how audits that don’t apply to you are removed. - [Audits](https://llmsrank.dev/docs/audits): every audit, what it checks, how it is scored, and how to fix it. - [Rubric changelog](https://llmsrank.dev/docs/changelog): each rubric version and what it changed, with the effect on our corpus. ## The standard llms.txt is a proposal by Jeremy Howard, first published on 2024-09-03 and revised as v2 on 2026-08-10. Its text lives at [llmstxt.org](https://llmstxt.org/), which also serves a [markdown version](https://llmstxt.org/index.md) and a [changes page](https://llmstxt.org/changes.html). Discussion and version history are in the [GitHub repository](https://github.com/AnswerDotAI/llms-txt). llmstxt.org lists three directories of published llms.txt files: - [llmstxt.site](https://llmstxt.site/) - [directory.llmstxt.cloud](https://directory.llmstxt.cloud/) - [llmstxthub.com](https://llmstxthub.com/) ## Guides - [About llms.txt](https://llmsrank.dev/learn/llms-txt.md): What llms.txt is, why it exists, where the file lives, its exact format, how agents use it, who publishes it, and what it is not, per the llmstxt.org v2 text. - [Writing an llms.txt file](https://llmsrank.dev/learn/writing-llms-txt.md): How to write an llms.txt file step by step, with complete examples for a small business and a docs site, the proposal’s guidelines and the mistakes we see most. - [Markdown page versions and link relations](https://llmsrank.dev/learn/markdown-pages.md): The llms.txt convention for markdown versions of pages (page.html.md or page.md), and the rel="alternate" and rel="describedby" links agents use to find them. - [llms.txt, robots.txt and sitemap.xml](https://llmsrank.dev/learn/llms-txt-robots-txt-sitemap.md): How llms.txt differs from robots.txt and sitemap.xml: purpose, readers, location, format and control, and why the proposal says a sitemap is no substitute. - [Does ChatGPT, Claude, Perplexity or Google read llms.txt?](https://llmsrank.dev/learn/does-ai-read-llms-txt.md): What each AI vendor has documented about reading llms.txt, with dates: Google Search ignores it; OpenAI, Anthropic and Perplexity have published nothing either way. - [llms.txt vs AGENTS.md vs robots.txt](https://llmsrank.dev/learn/llms-txt-vs-agents-md.md): Three files, three readers: robots.txt sets crawler access, llms.txt maps your content for AI agents, AGENTS.md instructs coding agents. Compared, with sources. - [How to check your llms.txt](https://llmsrank.dev/learn/check-llms-txt.md): Check an llms.txt for free: scan it on LLMSRank, work through the manual checklist step by step, or let an agent check it, and know what no check can tell you. - [llms.txt generators: what generated files miss](https://llmsrank.dev/learn/llms-txt-generators.md): What llms.txt generators do well and what they leave to you: link descriptions, curation, scope and routing, measured against the llms.txt Index of real files. - [llms.txt release notes](https://llmsrank.dev/learn/release-notes.md): What changed in the llms.txt proposal, newest first: v2 (2026-08-10), the optional BOM (2026-06-09) and v1 (2024-09-03), plus LLMSRank’s own rubric releases. - [The llms.txt Index: how 187 sites score](https://llmsrank.dev/learn/research.md): Aggregate results from 187 real llms.txt files scored with LLMSRank rubric v2.1: grades, the checks most sites miss, and every audit's pass rate. - [What we got wrong](https://llmsrank.dev/learn/what-we-got-wrong.md): A measurement product is only as good as its willingness to be wrong in public. Three times we published a number, found it was wrong, and said so — with the causes. - [The llms.txt year in review, 2026](https://llmsrank.dev/learn/llms-txt-2026.md): What changed for AI-readable websites in 2026, what our scan of 187 published llms.txt files found, and four things that move AI visibility we cannot measure. ## Frequently asked questions ### Who wrote the llms.txt standard? Jeremy Howard proposed it on 2024-09-03 and published v2 on 2026-08-10. It is a proposal, open for community input through its [GitHub repository](https://github.com/AnswerDotAI/llms-txt), not a standard from a standards body. ### Is the LLMSRank score part of the llms.txt standard? No. The proposal defines the file’s format and where it lives; it does not grade files. The score comes from LLMSRank’s rubric, which is documented at [/docs/method](https://llmsrank.dev/docs/method). ### How is LLMSRank different from the Lighthouse llms.txt audit? Lighthouse checks that an llms.txt file can be fetched, and marks a missing file as not applicable. LLMSRank reads the file and grades what is in it, along with the robots.txt, sitemap and agent files around it. ### Where can I see other sites’ llms.txt files? llmstxt.org lists three directories: llmstxt.site, directory.llmstxt.cloud and llmstxthub.com. ## Sources - [llmstxt.org — The /llms.txt file, v2 (modified 2026-08-10)](https://llmstxt.org/) - [llmstxt.org — Changes: v2 (August 2026)](https://llmstxt.org/changes.html) - [AnswerDotAI/llms-txt — the proposal’s GitHub repository](https://github.com/AnswerDotAI/llms-txt) - [Chrome for Developers — Lighthouse: llms.txt audit](https://developer.chrome.com/docs/lighthouse/agentic-browsing/llms-txt) ## Related - [About llms.txt](https://llmsrank.dev/learn/llms-txt.md) - [Writing an llms.txt file](https://llmsrank.dev/learn/writing-llms-txt.md) - [llms.txt release notes](https://llmsrank.dev/learn/release-notes.md)