What Is llms.txt? The Proposed Standard, Explained (2026)

If you have seen references to llms.txt (often mistyped as llm.txt) and wondered whether your site needs one, this guide covers what the file is, where it came from, how to create one, and, most importantly, an honest assessment of whether it does anything for your AI visibility today.

What is llms.txt?

llms.txt is a proposed standard for a plain markdown file placed at the root of a website (yoursite.com/llms.txt) that gives large language models a curated, machine-friendly overview of the site: a short description of what the site is, guidance on how to interpret it, and links to the pages that matter most. The idea is that AI systems have limited context windows and struggle with cluttered HTML, so a clean, curated index helps them find and use your best content. The proposal also describes an optional companion file, llms-full.txt, which contains the full text of key pages in one document.

Where it came from

The proposal was introduced in late 2024 by Jeremy Howard, co-founder of Answer.AI, as a community standard modeled loosely on conventions like robots.txt and sitemap.xml. Since then a number of developer-focused companies have published llms.txt files, and several documentation platforms can generate one automatically. Adoption has been most visible in technical documentation, where the use case is strongest: pointing a coding assistant at clean reference material.

llms.txt vs robots.txt vs sitemap.xml

The three files do different jobs. robots.txt controls access: it tells crawlers what they may and may not fetch. sitemap.xml is an exhaustive machine-readable list of URLs for indexing. llms.txt is curation: a human-authored selection of your most important content with context about what it means. It grants nothing and blocks nothing; it is a suggestion aimed at AI systems rather than search indexers.

The honest part: do AI companies actually read it?

Here is the assessment that matters. As of now, no major AI provider has publicly committed to consuming llms.txt. Google's search representatives have publicly expressed skepticism, comparing it to the old keywords meta tag, and server-log analyses shared by the SEO community show inconsistent, mostly negligible fetching of the file by AI crawlers. That does not make the file useless, it makes it speculative: a low-cost bet that the convention gains adoption, not a lever that changes your AI visibility today.

So should you add one?

Our honest recommendation: if you have a documentation-heavy or content-rich site, add one. It costs an hour, it cannot hurt, and if adoption arrives you are early. If someone is selling you llms.txt creation as an AI visibility strategy, walk away. It is a housekeeping item, not a strategy, and treating it as more than that is a red flag for the same reason 'we will submit your site to 500 search engines' was twenty years ago.

How to create an llms.txt file

The format is deliberately simple markdown: an H1 with the site or project name, a short blockquote summarizing what it is, then sections (H2s) grouping links to key pages, each with a one-line description. Keep it curated rather than exhaustive: the point is selection. Place it at the site root, keep it current when major content changes, and generate llms-full.txt only if your content is the kind an AI assistant would ingest whole, like API documentation.

What actually gets you cited by AI

If the goal behind your llms.txt interest is being mentioned and cited by ChatGPT, Gemini, Claude, and Perplexity, the levers with real evidence behind them are different: answer-first content structured for extraction, credible authors and entity signals models can resolve, presence on the third-party surfaces models already trust (community threads, review platforms, industry publications), and monthly measurement of how models actually answer questions about your category. That is the work of LLM SEO on your own site and LLM seeding across the wider web, the two halves of an AI search strategy. If you want to know where you stand today, our AI reputation management team starts with a free audit of what the major models currently say and cite about you.