TL;DR
- What it is: a Markdown file at
/llms.txtthat tells an AI agent what your site is and where the key pages are. - What it isn't: not robots.txt, not a sitemap. It blocks nothing and ranks nothing.
- How to write one: a title, a summary, and link lists. With a template.
- Does anything read it? Barely. On our site: 8 of 9,460 bot requests in two weeks. None from OpenAI, Google or Anthropic.
- Worth it? Yes for docs, APIs and agent products. No for rankings.
Does AI name your brand? Check your site.
Our site got 9,460 bot requests in two weeks.
8 of them were for llms.txt.
That's the honest starting point for this guide.
llms.txt is cheap to write. It's useful in a few cases. And it won't get you cited by ChatGPT.
Here's what it is, how to write one, and what three data sets say about who reads it.
What llms.txt is
A plain Markdown file. At the root of your site. Named llms.txt.
It tells an AI agent two things:
- What this site is.
- Where the pages that matter are.
That's it.
Jeremy Howard proposed it on 3 September 2024. It's a proposal. Not a web standard that anyone has to follow.
The idea: an agent can't read your whole site. So give it a short guide instead.
The proposal says the file is mainly meant for when an agent answers a question, not for training.
Note the name. It's llms.txt, with an s. Not llm.txt.
llms.txt vs robots.txt vs sitemap
People mix these up. They do three different jobs.
| File | Job | Who reads it |
|---|---|---|
robots.txt | Rules. What a crawler may fetch | Crawlers |
sitemap.xml | A list of every URL you want indexed | Search engines |
llms.txt | A short guide to the site | Agents, sometimes |
The big one: llms.txt blocks nothing.
You can't use it to keep AI out. That's robots.txt.
And you can't use it to get AI in. Nobody promised that.
What goes in an llms.txt
Four parts. Only the first is required.
The proposal's own mock example:
# Title
> Optional description goes here
Optional details go here
## Section name
- [Link title](https://link_url): Optional link details
## Optional
- [Link title](https://link_url)
# Title: the name of the site or project.> Summary: a short blockquote. The key facts.- Details: plain paragraphs or lists. No headings.
## Sections: lists of links. Each link can have a note after a colon.## Optional: a special section. Links an agent can skip if it's short on space.
A real example: ours
Here's the top of our own llms.txt.
Three things it does that a sitemap can't:
- It says what the product is. In one paragraph.
- It says how to use it. "Fetch SKILL.md first. It has the MCP URL, tool order, and rules."
- It says what not to do. "Do not invent ranks."
So ours isn't a page index. It's a front door for agents.
That's the use case where llms.txt makes sense. A product that agents work with: docs, an API, an MCP server, skills.
How to write an llms.txt
1. Name the site. One H1.
2. Summarize it in one paragraph. What you sell. Who it's for. One or two hard facts, like your price.
3. Link the pages that matter. Not every page. Pricing, docs, your best guides. I'd keep it to 10 to 20 links.
4. Save it as /llms.txt. At the root. Served as plain text or Markdown.
5. Check your logs after a month. See who asked for it. More on that below.
A template:
# Acme Invoicing
> Acme is invoicing software for freelancers. Plans start at $12 a month.
> It sends invoices, tracks payments and exports to accounting tools.
Acme does not do payroll. Prices on this page are in US dollars.
## Product
- [Pricing](https://acme.example/pricing): plans and limits
- [Features](https://acme.example/features): what each plan includes
## Docs
- [Getting started](https://acme.example/docs/start): setup in five steps
- [API reference](https://acme.example/docs/api): endpoints and auth
## Optional
- [Blog](https://acme.example/blog)
Three rules:
- Keep facts current. A stale price in llms.txt is worse than no file.
- Don't paste your sitemap. A list of 2,000 URLs helps nobody.
- No secrets. The file is public.
Do you need a generator? Not really. It's short enough to write by hand.
Does any AI read llms.txt
Here's where most guides go quiet.
Three data sets. One of them is ours.
Ahrefs: 97% of files got zero requests
Ahrefs looked at 137,210 domains with traffic in May 2026.
- 28% publish an llms.txt.
- 97% of the valid files got zero requests that month.
- Of the bots that did read one, 77% weren't from AI tools.
Slackbot fetched more llms.txt files than PerplexityBot did. People share the link in chat. The preview bot fetches it.
EZY: 7 fetches against 3,990
EZY put llms.txt on 83 sites and watched the logs for 12 weeks.
- OpenAI: fetched llms.txt 7 times. And robots.txt 3,990 times.
- Perplexity: fetched llms.txt 0 times. And robots.txt 775 times.
Our own logs: 8 of 9,460
Now ours. aiseotracker.com, 3 to 17 September 2026. IP-verified bot requests only.
(Our bot log has no data after 17 September. So this is two weeks, not a month.)
- 9,460 bot requests in total.
- 493 for robots.txt.
- 8 for llms.txt.
Who asked for it? Meta's crawler 5 times. Amazon's 2 times. PerplexityBot once.
OpenAI: zero. Google: zero. Anthropic: zero.
So what do AI assistants fetch?
Your pages.
In the same two weeks, AI assistants made 686 requests to answer a user. ChatGPT alone: 565.
The top target was our homepage. 180 fetches by ChatGPT.
Then leaderboard pages. Then blog posts.
Plain HTML pages. Not llms.txt.
One site. Two weeks. But it matches the other two.
What Google says
Google's guide to AI features is direct. Google Search doesn't use these files. Keeping one neither helps nor harms your visibility there.
Is an llms.txt worth it
Yes, if agents work with your product.
Docs. An API. An MCP server. A developer tool.
A coding agent that lands on your site can use a short guide. And a person can paste the URL into a chat to give the model context.
No, if you want rankings or citations.
The logs say AI search bots mostly don't fetch it.
What gets you cited is on your normal pages: the answer in plain text, the facts an engine can quote, and a spot on the lists engines already cite. How to get cited in AI answers covers that.
My take: write one if it takes you an hour or less. Then move on to the pages.
Want agents to read your pages cheaper? That's a different technique. See Markdown for agents.
FAQ
Is llms.txt a ranking factor?
No. Google says its Search doesn't use the file.
Does ChatGPT use llms.txt?
Not in the logs we have. OpenAI's bots made zero llms.txt requests on our site in two weeks of logs. In EZY's study of 83 sites, 7 in 12 weeks.
ChatGPT does fetch normal pages. A lot.
Where do I put the file?
At the root of your domain: https://yoursite.com/llms.txt.
Is it llm.txt or llms.txt?
llms.txt. With an s.
Can llms.txt block AI crawlers?
No. It has no rules in it. Use robots.txt for that.
Should I link every page?
No. Link the pages an agent needs to understand you. The proposal has an "Optional" section for the rest.
Does AI SEO Tracker have an llms.txt generator?
No. We don't have one.
Our free tools are the Page Inspector, keyword research and the AI share button generator.
