llms.txt Without the Myths: What It Actually Does and Whether You Need It
llms.txt is a proposed convention for publishing a concise summary of a website's content and important links in Markdown format. It is not a generally binding standard, nor a guaranteed AI-visibility ranking factor. Google explicitly states that it does not use this file for Search, including its generative features.
What llms.txt is
The proposal, published at llmstxt.org, recommends placing the file at:
https://yourdomain.com/llms.txt
It's meant to help language models work with a website at inference time. It typically contains:
- the project or company name;
- a brief summary;
- supplementary context;
- a list of the most important pages with descriptions;
- an optional section of less important links.
Example:
# SEO Radar
> A tool for checking a website's technical SEO and visibility
> in search engines and AI answers.
SEO Radar analyzes a website's accessibility, content, structured data,
entities, and other signals relevant to SEO and AI search.
## Documentation
- [How the audit works](https://example.com/how-it-works): Scope and methodology of the check.
- [Knowledge base](https://example.com/knowledge-base): Practical guides on SEO and AI.
## Services
- [SEO audit](https://example.com/seo-audit): Technical and content review of a website.
- [AI visibility](https://example.com/ai-visibility): Measuring mentions, citations, and recommendations.
What llms.txt is not
It is not:
- a replacement for robots.txt;
- a sitemap for search engines;
- an indexing directive;
- a guarantee that AI will use the content;
- a secret ranking factor;
- a way to fix a low-quality website;
- a standard automatically supported by ChatGPT, Google, Gemini, and Perplexity.
The name might suggest it's an equivalent of robots.txt, but the two files play different roles. Robots.txt has an established function in controlling crawling. llms.txt offers curated content for tools that voluntarily choose to process it.
What Google says
Google's official guidance for generative AI search states that:
- llms.txt is not required to appear in AI Overviews or AI Mode;
- Google Search does not use this file;
- creating it will not positively or negatively affect visibility or ranking in Google.
This is an important correction to the common marketing claim that llms.txt is one of the fundamental technical GEO factors.
What OpenAI says
OpenAI's public documentation for publishers emphasizes primarily:
- allowing
OAI-SearchBot; - allowing its published IP ranges;
- public availability of content;
- the ability to track referral traffic from ChatGPT.
llms.txt is not listed in this guidance as a condition for inclusion in ChatGPT Search. That doesn't mean no tool can use it. It means no officially confirmed effect on ChatGPT Search can be attributed to it.
When it can make sense
The file can be useful when:
- you run extensive documentation;
- you want to offer a concise entry point for developer and agentic tools;
- you're able to keep it automatically up to date;
- it links to canonical, high-quality pages;
- the cost of creating and maintaining it is minimal;
- the specific service you use declares support for it.
It's a supplement, not a priority ahead of indexing, content, or authority.
When you'd rather not create it
Don't create it just to tick a box on a GEO checklist. It can actually be operationally harmful if it:
- contains outdated prices and services;
- links to redirected or defunct URLs;
- contradicts information on the website;
- exposes non-public pages;
- is automatically generated without review;
- creates another source of truth that nobody maintains.
llms-full.txt
The proposal also allows for a more extensive llms-full.txt file, which can contain more textual content. For a large website, though, this quickly runs into problems with staleness, size, and duplication. Before deploying it, you need to know the specific consumer and purpose.
Recommended order of priorities
- Fix crawling, indexing, and HTTP errors.
- Ensure high-quality canonical pages.
- Sharpen content and entity information.
- Add structured data where it matches the content.
- Allow individual AI crawlers based on a business decision.
- Only then create an llms.txt as a low-cost extra.
How to check the file
- Does it return a 200 status?
- Is it accessible without a redirect?
- Does it have a text or Markdown MIME type?
- Does it use absolute canonical URLs?
- Does it avoid exposing non-public information?
- Do the links work?
- Do the descriptions match the actual content?
- Does it have a designated owner and an update process?
Verdict
llms.txt can be deployed if it's easy to maintain and you have a concrete use for it. It isn't, however, a proven way to increase citations in Google or ChatGPT, and shouldn't be presented as one. Priority belongs to the publicly documented fundamentals: accessibility, quality content, correct identity, trustworthiness, and measurement.
Frequently asked questions
Will llms.txt help in Google?
According to current Google Search documentation, no. Google ignores the file for its generative features.
Does it replace sitemap.xml?
No. A sitemap helps search engines discover URLs. llms.txt is a curated text overview for tools that support the proposed convention.
Does it have to be at the site root?
The proposal specifies the path /llms.txt at the root of the domain.