llms.txt is a genuine proposal for giving agents a curated route through a site. It is not a search ranking or AI citation lever. Google says it ignores the file. Publish one if you have a defined consumer for it, and govern it like public documentation.
The answer
No evidence located shows that publishing llms.txt causes more public AI citations or recommendations. That is narrower than "it does nothing", and the difference matters, because there is now published evidence that agents given the file do read and follow it.
What exists: a real proposal, visible adoption, near-zero passive consumption in the largest log study, a negative passive-discovery result in one small experiment, traced evidence that agents which reach the file do use it, an explicit statement from Google that Search ignores it, and no documented citation benefit from any answer engine.
What does not exist: any controlled evidence of benefit, from anyone, so far.
Six events, not one question
Almost every argument about this file collapses a chain into a single verdict. Evidence for an earlier step does not establish a later one:
| Event | What would count as evidence | Status |
|---|---|---|
| A site publishes the file | A valid file returned from /llms.txt | Well documented |
| A crawler requests it | Verified server-log requests | Rare in the largest study |
| An agent parses it | Observable use of its structure | Demonstrated in traced runs |
| It helps the agent navigate | Agent follows a listed page or answers from it | Demonstrated in traced runs |
| Content is cited in an answer | Citation attributable to the file | No evidence located |
| Visibility or revenue changes | Controlled before and after | No evidence located |
A published file can exist without being fetched. A crawler can fetch it without any product using it. An agent can use it when explicitly handed the file without that affecting public search at all.
What the proposal says
The specification calls itself a proposal. It recommends a concise Markdown overview, links to detailed Markdown pages, and optional HTML or HTTP link relations advertising the file. It is designed to help agents navigate information. It is not a crawl directive like robots.txt.
The only required format element is an H1. A blockquote summary and grouped link lists are recommended, not required. So publishing a valid file demonstrates one thing: that a site has made an agent-facing map available.
What Google says
Google addresses the file by name, and the wording is not ambiguous:
Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.
Its separate AI features guidance says you do not need to create machine readable files, AI text files, markup or Markdown to appear in Google Search including its generative capabilities. Checked 2 September 2026.
There is an apparent contradiction worth resolving. Chrome's Lighthouse includes an agentic-browsing audit that recognises llms.txt as an emerging optional convention. That is not Google Search contradicting itself. Chrome's agent-navigation audit and Google's ranking systems are different products testing different hypotheses, and only one of them is about ranking.
OpenAI and Anthropic both publish llms.txt indexes for their own developer documentation. That demonstrates documentation use. It says nothing about how their consumer answer products discover or rank third-party business sites. Those are different systems.
What the logs show
Ahrefs examined 137,210 domains in its Web Analytics product during May 2026.
- 28% published a genuine llms.txt.
- 97% of those files received zero traffic that month. Nothing fetched them at all.
- Of requests that did arrive, 96% came from bots, and 77% of those bots were not AI tools.
- Roughly 1.1% of file requests came from AI retrieval bots.
Ahrefs publishes its own caveat, which we will repeat rather than bury: its analytics customers skew more technical and SEO-aware than the web generally, so 28% adoption should be read as an upper bound. It also did not test whether files were well-formed against the specification.
A request is not proof of parsing, and user agents can be spoofed. The narrow conclusion the data supports: in that dataset, publication was far more common than observed consumption.
What traced agent runs found
AgentReady, an open standard published by ora.ai with Vercel and Mintlify, reports 1,033 traced agent runs. It sells nothing, which in this field is worth stating plainly.
- 329 runs reached and read an llms.txt file.
- 120 of those runs, about 36%, drew their final answer from its content.
- 117 runs went on to fetch a page the file listed, roughly one in three.
- A separate controlled experiment ran 190 fetchability probes across one documentation site under 19 configurations.
That is direct evidence of parsing and navigation, which is what rungs three and four ask for. We reviewed the published method and counts rather than reprocessing the raw traces.
It does not climb the ladder further. These are agent sessions on documentation tasks, not consumer search answers, and the runs reached the file rather than discovering it unprompted. A file that helps an agent complete a task is not evidence that publishing one gets a business recommended.
What a controlled test found
Reboot ran the clearest passive-discovery experiment we located. It added four new orphan pages to each of two existing sites, referenced only from root llms.txt files. Over three months, AI bots continued crawling other pages and did not fetch the files or the eight test pages.
That is a negative result under those specific conditions. Two domains and eight generic pages is a small test, and it did not examine explicit entry-point use, rankings or citations. It is evidence against passive discovery, not proof of a universal zero.
SE Ranking separately analysed nearly 300,000 domains in November 2025 and detected no association between file presence and domain-level AI citation frequency. Absence of a detected association is not proof of no effect, and both vendors sell AI tracking tools, which in this case cuts against their own products.
How common it actually is
We checked 48 active branding, design and web agency sites on 1 September 2026. Eighteen returned a valid minimum-format file. That is 16 of 30 in the India cohort and 2 of 18 internationally.
The files ranged from 191 to 8,745 words, median 585.5. Some were concise maps, others looked machine-generated.
The full method, every denominator and the aggregate dataset are published separately. It is an adoption finding in a selected commercial sample, and it measures only the first rung of the ladder above.
Why we stopped saying it is harmless
This page used to say the file costs nothing and does no harm. We have changed that, and the reasoning is worth showing.
Security researcher Alon Hertz reported scanning thousands of llms.txt and llms-full.txt files and finding unclaimed package, domain and subdomain references inside instruction-shaped documentation, with coding agents following some of those references in controlled configurations.
That account is self-published, does not include a complete per-model raw dataset, and concerns privileged coding agents rather than consumer answer engines. We are not treating it as settled. But it is enough to defeat an absolute claim, and the absolute claim was ours.
"Does no harm" was unfalsifiable. An agent-facing file is public documentation, and public documentation has a maintenance surface:
- Keep it in version control with a named owner and a review date.
- Avoid executable instructions unless there is a defined technical consumer.
- Verify every package name, domain and subdomain it references.
- Review anything a plugin generated automatically before leaving it live.
- Never include secrets, private endpoints or internal operating instructions.
Our own file previously carried the "does no harm" wording. It now describes the file as low-cost but not risk-free, and states how we govern it. The change is recorded here rather than made quietly.
Should you publish one
| Situation | Decision |
|---|---|
| Ordinary service-business site, no defined agent consumer | Optional, low priority |
| Important pages blocked, unindexed or thin | Fix those first |
| A documentation platform or customer agent consumes it | Publish and govern it for that use |
| The reason is "Google AI will rank us" | No. Google says it ignores the file |
| A plugin generated a long file automatically | Review before keeping it live |
| It contains package, domain or execution instructions | Security and ownership review first |
We publish one. It is documentation hygiene, not a tactic, and the file says so in its own text. We do not sell it as a deliverable and would tell you not to buy one.
The file is the easy part. The difficult work is still producing evidence worth finding rather than markup that looks like effort, and keeping it technically accessible. If you want to know whether anything you publish is moving, measure it against a dated baseline.