On 17 October 2026 we requested /llms.txt and /llms-full.txt from the 10,000 most popular sites on the web, then read every file we got back against the llms.txt format.
The short answer: 15.2% of sites serve an llms.txt file, about one in seven. A quarter of those files don't follow the format, and only 2.5% of sites also ship the longer llms-full.txt.
Check your own site with the free AI crawler checker, which looks for an llms.txt as well as reading your robots.txt. If you don't have one yet, the llms.txt generator drafts one from your home page and sitemap.
What we measured
- Sites: the Tranco top 10,000 list, downloaded the day of the run.
- Files:
/llms.txtand/llms-full.txton the bare domain, or onwww.when the bare domain didn't answer. - What counts as present: a 2xx answer with a non-empty body that is not an HTML page. Many servers answer every unknown path with their home page or a "not found" page and a 200 status. Those don't count.
- What counts as following the format: the file starts with a
#title and contains at least one markdown link. That is the minimum the llmstxt.org proposal describes. We also recorded whether it has a>summary line, how many##sections and links it has, its size and its content type.
6,502 sites gave an answer we could read. The other 3,498 were left out of every percentage: they didn't answer at all, refused us with a 401, 403 or 429, or returned a server error. Many Tranco entries are CDN, API and tracking domains with no website behind them. Every percentage below is out of the 6,502 unless it says otherwise.
Finding 1: about one site in seven has an llms.txt
| Sites | Serve llms.txt | Serve llms-full.txt |
|---|---|---|
| All (6,502) | 15.2% (988 sites) | 2.5% |
| Top 1,000 (608) | 18.3% | 3.0% |
| Rank 1,001 to 10,000 (5,894) | 14.9% | 2.5% |
The most popular sites are a little more likely to have one, but the gap is small. llms.txt is still an early-adopter file, even at the top of the web.
Every site with an llms-full.txt also had an llms.txt. Nobody ships the long version alone.
Finding 2: a quarter of the files don't follow the format
Of the 988 sites with an llms.txt:
| Check | Share of files |
|---|---|
Start with a # title |
83.3% |
Have a > summary line near the top |
75.9% |
| Have no links at all | 15.2% |
| Follow the basic format (title and at least one link) | 74.1% |
Served as text/plain or text/markdown |
97.5% |
| End up on a different host after redirects | 8.4% |
A file with no links is the most common miss. It tells a model what the site is but gives it nothing to read next, which is the point of the file. The top 1,000 do better: 85.6% of their files follow the format, against 72.6% for the rest.
About one file in twelve is served from a different host after redirects, usually a docs subdomain or a separate docs platform. That's allowed, but it means the file lives somewhere the main site's team may not look at.
Finding 3: most files are short lists of links
| Measure | All files | Top 1,000 |
|---|---|---|
| Median links | 33 | 62 |
Median ## sections |
7 | 9 |
| Median size | 6.9 KB | 12.2 KB |
| Over 100 KB | 7.0% | 10.8% |
The typical llms.txt is a few kilobytes: a title, a summary, a handful of sections and a few dozen links. About 7% run past 100 KB, which usually means the whole site map was pasted in. That defeats the purpose of a short index, and llms-full.txt is the intended home for long content.
Finding 4: SEO plugins are writing some of them
8.2% of the files say what generated them. Among the ones that name a tool, the most common are Yoast (44 sites), Shopify (10), Rank Math (9) and All in One SEO (8). Docs platforms such as Fern and GitBook appear a few times.
Treat these counts as a floor. A generator that doesn't leave a note in the file isn't counted, and we only looked at the first few kilobytes.
What this study does not tell you
- Whether anyone reads the file. We measured what sites publish, not what any AI product fetches or uses. Having one is cheap, but it is a bet, not a ranking factor.
- Whether the links work. We counted links. We didn't follow them.
- Quality. A file can follow the format and still describe the site badly.
- One day. Adoption is moving fast. We will rerun the study and publish the comparison.
What is an llms.txt file?
It's a markdown file at /llms.txt that gives language models a short guide to a site: a title, a one-paragraph summary, and lists of links to the pages that matter most, grouped under headings. The format was proposed at llmstxt.org in 2024. llms-full.txt is an optional companion that holds the full text of those pages in one file.
Do I need an llms.txt?
Not to rank in search. It may help AI assistants and coding agents find the right pages on your site, and it costs little to add. If you add one, keep it short, start it with a # title and a > summary, and link to the pages you would want someone new to read first.
How do I make one?
Paste your URL into the llms.txt generator. It builds a draft from your home page and sitemap that you can edit, copy or download. It's free and needs no signup.
Method and code
The survey ran on a GitHub Actions runner, 64 connections at a time, with the user agent SEOScoreAPI-Research/1.0. It made one request for /llms.txt and one for /llms-full.txt per site and did nothing else. The run took 15 minutes. This is the companion to our study of which AI crawlers the top 10,000 sites block, run on the same list the same day.
Once AI assistants can read your site, AI citation tracking shows whether they actually mention your brand.