18 of 48 agency sites published llms.txt. Sixteen days later, 20 did.

The finding

In a purposive sample of 48 active branding, design, creative and web agencies checked on 1 September 2026, 18 sites returned a valid minimum-format root llms.txt.

The cohort gap is large and we are not going to explain it, because we did not test an explanation. It could be selection, site vendors, CMS plugins, timing, or something we did not measure. Describing it is honest. Attributing it would not be.

Six events, not one

Most arguments about llms.txt collapse a chain of separate events into a single yes or no. Evidence for an earlier link says nothing about a later one:

• A site publishes the file. This study measures this one.
• A crawler requests it.
• An agent parses it. Demonstrated in traced runs.
• The file helps that agent navigate. Demonstrated in traced runs.
• Content gets cited in a public answer.
• Visibility or business outcomes change.

We measured the first. Rungs three and four have since been demonstrated in traced agent runs published by AgentReady, which we had not found when this study went up. The last two remain open, and conflating all six is how a documentation convention became a billable GEO deliverable.

What we checked

For each of the 48 agencies, a read-only collector requested the public homepage, root robots.txt, the first sitemap advertised in robots.txt (or root sitemap.xml if none), and root llms.txt. Six-site concurrency, 18-second timeout, user agent BORVISResearch/1.0 (+https://getborvis.com/). No forms submitted, no accounts created, no restricted interfaces touched.

A file counted as valid only when the route returned HTTP 200, the response did not look like HTML or a soft 404, and the body contained at least one Markdown H1, which is the proposal's only required element. Five positive files were manually sampled before the full validation pass.

Three homepages returned a timeout, a 502 and a 403. Those are time-stamped responses on one day, not claims that the businesses or sites are inactive.

The files were not alike

The range is more informative than the adoption number.

A file forty times longer than another is not forty times more useful to an agent. Presence tells you nothing about accuracy, maintenance, or whether any consumer wants that much context. Several of the longest files looked generated rather than curated.

Robots told a clearer story

Among the 44 sites with a valid robots file, the access picture was more concrete than the optional map.

• None explicitly blocked OAI-SearchBot at the root.
• Four explicitly allowed it.
• Forty relied on wildcard rules that did not block the root.
• Four explicitly blocked GPTBot while leaving OAI-SearchBot unblocked through wildcards.

OpenAI documents these as separate controls: OAI-SearchBot concerns inclusion in ChatGPT search, GPTBot concerns potential model training, and ChatGPT-User handles user-triggered visits.

That last group of four is interesting. Blocking training while leaving search open is a coherent policy, and it is the position we would defend. But we did not ask those site owners what they intended, so we are not going to claim it was deliberate.

What the largest log study found

Our study cannot see whether anything fetched these files. One study can.

Ahrefs examined 137,210 domains in its Web Analytics product for May 2026. In that dataset, 28% published a genuine llms.txt, and 97% of those files received zero traffic that month. Of the requests that did arrive, 96% came from bots, 77% of those bots were not AI tools, and roughly 1.1% came from AI retrieval bots.

Ahrefs states its own limitation, which is worth repeating: its analytics customers skew more technical and SEO-aware than the web at large, so 28% should be treated as an upper bound. They also did not check whether files were well-formed against the specification, which is the check we did run.

Put the two together and the picture is coherent. Publication is common. Consumption is not.

Sixteen days later

On 17 September 2026 we re-requested the same 48 endpoints with the same frozen screen. Twenty passed, up from 18.

All 18 baseline files still passed. Two endpoints that returned 404 in September now return a valid file. None stopped passing.

Counting status codes would have overstated it

Twenty-three endpoints returned HTTP 200. Only 20 passed the screen. The gap is the finding.

• One 200 response is an ordinary HTML page, not an llms file at all.
• Two 200 responses are short directive-style text without the tested heading.
• One further 200 response looks like a genuine agency index, but its opening heading is indented, so the frozen rule excludes it. Counting it would give 21 of 48 rather than 20.

We are reporting both numbers rather than quietly relaxing the rule, because a screen you adjust after seeing the results is not a screen.

What moved is availability. Nothing here says an AI product read any of these files, or that the two agencies who added one will be recommended more. Sixteen days apart is two observations, not monitoring, and newly observed availability is not an installation date.

Wave two data: agency-llms-txt-study-wave2-2026-09-17.json. Same aggregate-only rule as the baseline, with no agency names or URLs.

Should you publish one

Google's position is the least ambiguous input available:

Doing so will neither harm nor help your site's visibility or rankings in Google Search, as Google Search ignores them.

We publish one on this site. Our reasoning is on the record, and the file itself says it is documentation hygiene rather than a tactic. Do not buy it as a standalone GEO deliverable.

Limits and data

• Purposive commercial sample of 48 agencies, not a random or representative industry sample.
• Two collection dates, 16 days apart. That is a repeat observation, not monitoring, and HTTP results are a snapshot that changes.
• Presence is an adoption observation. It is not evidence of indexing, ranking, citation, recommendation or traffic.
• The cohort difference has no tested explanation.
• A robots rule is a published preference, not proof of crawler behaviour.
• Route and schema fields are homepage-source observations, not full audits. Not detected is not the same as does not exist, since rendering and unusual naming defeat the detector.

Both aggregate datasets are published so the numbers can be checked: the 1 September baseline and the 17 September wave. It contains every numerator, denominator and definition used above, plus the interpretation limits.

It does not contain the agency names or URLs. Those sites were selected through our own commercial research, and publishing a list of named businesses beside their technical shortcomings is not something we are willing to do. The aggregate is enough to check the arithmetic, which is the part that matters.

We observed adoption, not impact. Anyone claiming the second from evidence like ours is overreaching.