# Encoding and parsing UTF-8 and parses as markdown **In short:** Save the file as UTF-8 plain markdown that starts with a # title and lists links as - [name](url); the parser needs one of the two for full marks. - Category: Retrievability & Access · weight 15 of 100 within the category - Retrievability & Access is worth 15 in every profile (points of 100, by scoring profile) - Audit ID R4 (as shown on reports) · rubric v2.1 ## What it checks Whether the file is valid UTF-8 and parses cleanly as markdown. ## Why it matters Agents read the file as markdown. Encoding errors and broken markdown turn headings and links into noise the model has to guess at. ## When it applies Always, for any llms.txt that exists. An audit that doesn’t apply, or that we couldn’t verify, is removed from your score and its weight shared among the Retrievability & Access audits that do apply. It is never scored as a zero. ## How it’s scored 1.0 when the parser finds an H1 (# Name) or at least one markdown link list item (- [name](url)) · 0.5 when it finds neither. > **Measurement.** Today this audit checks only that the file parses into something recognisable. It doesn’t check that the bytes are valid UTF-8 (invalid bytes are decoded as replacement characters) or count parse errors; scaling the score by parse errors is planned. ## How to fix 1. Save the file as UTF-8 (a BOM is allowed but not needed). 2. Use plain markdown: # for the title, ## for sections, - [name](url) for links. 3. Remove HTML tags, templating leftovers and unclosed brackets. ## Frequently asked questions ### Is a byte-order mark (BOM) at the start of llms.txt allowed? Yes. llms.txt v2 lists an optional BOM as the first element of the file, and we strip it before parsing, so it never hides your H1. It isn’t needed for UTF-8, so leaving it out is simplest. Sources: [llms.txt (v2, Aug 2026)](https://llmstxt.org/) · [RFC 3629 §6: Byte order mark](https://www.rfc-editor.org/rfc/rfc3629.html#section-6) ### What does the Encoding and parsing audit actually check today? We decode the file as UTF-8 and parse it as markdown. Today it scores 1.0 when the parser finds an H1 (`# Name`) or at least one markdown link list item (`- [name](url)`), and 0.5 when it finds neither; the rubric’s per-error scaling isn’t applied yet. Sources: [Encoding and parsing](https://llmsrank.dev/docs/audits/encoding-and-parsing.md) · [LLMSRank method](https://llmsrank.dev/docs/method.md) ### My file was saved as Windows-1252 or Latin-1. Does that matter? Yes, for readers: we and most agents decode the bytes as UTF-8, and non-ASCII characters in another encoding (curly quotes, accented names, €) turn into replacement characters. Re-save the file as UTF-8 and declare `charset=utf-8` in the Content-Type. Sources: [RFC 3629: UTF-8](https://www.rfc-editor.org/rfc/rfc3629.html) · [Encoding and parsing](https://llmsrank.dev/docs/audits/encoding-and-parsing.md) ### Can I start llms.txt with an HTML comment such as a version stamp? Yes. Leading `` comments are skipped before we look for markup or parse the file, so `` above your H1 is fine. Any other tag at the top, such as `` or `
`, makes the response count as a web page, and it is not scored as an llms.txt. Sources: [llms.txt (v2, Aug 2026)](https://llmstxt.org/) · [Encoding and parsing](https://llmsrank.dev/docs/audits/encoding-and-parsing.md) ### In your research, how many llms.txt files parse cleanly? Of the 156 tracked sites, 98.7% (154) earn full marks, with a mean score of 0.994. The two that don’t have a file where the parser finds neither an H1 nor a single markdown link. Sources: [LLMSRank method](https://llmsrank.dev/docs/method.md) · [Encoding and parsing](https://llmsrank.dev/docs/audits/encoding-and-parsing.md) See also: [all audits](https://llmsrank.dev/docs/audits.md) · [how scoring works](https://llmsrank.dev/docs/method.md)