Definition: llms.txt is an emerging web convention — proposed in 2024 and still gaining adoption — in which a plain-text or Markdown file is placed at the root of a website (e.g. example.com/llms.txt) to help large language models and AI assistants identify and prioritise the site’s most trustworthy, structured content. The file typically contains curated links to documentation, FAQs, product pages, APIs, and policies, with brief descriptions of each.
Why It Matters: As AI assistants and chatbots increasingly synthesise answers from web sources rather than directing users to click links, llms.txt is positioned as a way for site owners to influence which content AI systems reference. It complements — but does not replace — robots.txt (which controls crawler access) and sitemaps (which list all indexable URLs). For SEO and content teams, monitoring how AI systems cite and represent brand content is becoming a distinct discipline, and llms.txt is one early mechanism for shaping that. Adoption among AI providers is still uneven as of early 2026.
How It Works: The llms.txt file follows a simple Markdown structure: a site name and description at the top, followed by sections of curated links with brief annotations. Unlike sitemaps, it is selective — intended to highlight the most useful pages for AI consumption rather than enumerate every URL. Some content management systems and SEO plugins have begun auto-generating llms.txt files. AI providers that choose to respect the file can use it to prioritise high-quality sources when constructing answers, though there is no standardised enforcement mechanism equivalent to robots.txt’s well-established crawler compliance norms.
Related: Sitemap, Retrieval-Augmented Generation, Search Engine Optimization, Structured Data, AI Overviews