A proposed Markdown-based file convention providing curated website information and links for compatible AI tools; not a universal access or ranking requirement.
Llms.txt is a proposed Markdown-based convention for giving compatible AI tools concise information and links about a website. It can help organize access to documentation or other resources for systems that choose to use it. It is not the same file as robots.txt and does not serve the same purpose.
The proposal describes an informational guide, not a universal access-control or ranking mechanism. Publishing the file does not establish that every provider reads it, obeys it or selects the site for an answer. Google’s AI search guidance says new AI text files are not required for inclusion in its AI features.
If a site uses the convention, the contents should be accurate and maintained. A curated index can become misleading if it points to removed documents or describes capabilities the business no longer offers. The ordinary public pages still need to be useful and accessible; the file should not become a second contradictory version of the business’s information.
Takeaway: Treat llms.txt as an optional informational convention for compatible tools, not a guaranteed visibility upgrade. Keep its claims and links consistent with the actual website.
Example
A software provider creates a concise index of its current installation and integration documentation. When an API version is retired, it updates the index and relevant pages together. The file makes navigation more organized for compatible readers, but the provider does not claim it guarantees citations in every AI product.
What to check
Check the proposal’s current format and the tools you actually expect to consume it. Verify every linked resource and keep private material out of the public file. Do not confuse a link index with permission to crawl, a training opt-out or a substitute for well-structured public content.
For more detail, see Llms.txt proposal.