Direct answer
Add it, but for the right reason. llms.txt is a proposed root-level file listing your most important pages for large language models — put forward by Jeremy Howard at Answer.AI in September 2024. No major answer engine has publicly committed to reading it, so nobody can honestly promise a visibility return. It costs an afternoon. The reliable payoff is the exercise: deciding which twenty pages matter and describing each in one plain sentence.
The pitch arrives as urgency: add this file or the models will skip you. That framing is wrong in both directions. It overstates what the file currently does and understates the only benefit that doesn’t depend on anyone else’s roadmap.
What it actually is
The idea, proposed by Jeremy Howard of Answer.AI in September 2024, borrows the shape of robots.txt: a single file at the root of the site, plain text, listing the pages that matter with one line explaining each. Where robots.txt tells a crawler what it may take, llms.txt tells a model what is worth reading. The intent is to hand over a clean map instead of leaving a machine to reconstruct one from navigation and links.
The honest state of adoption
No major answer engine has publicly committed to reading it. That is the sentence any vendor selling llms.txt as an AI-visibility service should be made to say out loud. It may become a standard; conventions have spread from less. But today, anyone quoting a visibility gain from adding one is describing a hope, not a measurement.
What is verifiable: the proposal originates with Jeremy Howard at Answer.AI, published in September 2024, and is documented publicly at llmstxt.org. What is not verifiable is engine adoption — none of the major answer engines has published a commitment to consume the file. Both halves of that belong in any recommendation to add one.
Why it’s still worth an afternoon
Because of the cost on one side and what the writing exposes on the other. Producing the file means choosing the twenty pages that actually carry your commercial argument and describing each in one plain sentence a stranger would understand. Most firms cannot do that quickly. The struggle is the finding: if you can’t say in a line what a page is for, a machine reading it has the same problem, and so does a buyer.
If you can’t describe the page in one plain sentence, the machine reading it has the same problem you do.
| The claim being sold | What holds up | |
|---|---|---|
| Effect on citations | Engines will start quoting you | Unproven; no public engine commitment |
| Cost | Sold as a service | An afternoon, in-house |
| Risk | Falling behind without it | Effectively none either way |
| Reliable benefit | Not mentioned | Forces a plain-language map of your own site |
What not to pay for
Don’t buy it as a package, and don’t let it be the deliverable that stands in for the harder work. A file listing pages nobody has a reason to quote changes nothing, in the same way that markup on a thin page changes nothing. Sequence it after the material is worth pointing at — and if you are weighing where the budget genuinely moves the needle, that comparison sits in SEO versus AEO and which one gets you chosen.
The proportionate answer
Write it yourself, keep it current, expect nothing from it, and take the clarity you got from writing it as the return. That is a reasonable trade for an afternoon. It is not a reasonable line item on an invoice, and it is not a strategy — the actual costs of that work are set out in what answer-engine optimisation actually costs.
- List the twenty pages that carry your commercial argument — no more.
- Write one plain sentence per page that a stranger to your industry would understand.
- Note which pages you couldn’t describe in one sentence. Fix those pages first.
- Keep the file current when pages change, or delete it — a stale map is worse than none.
- Don’t pay a vendor for it, and don’t accept it as the deliverable in place of harder work.
If you’d rather know which of your pages an engine can actually use before you write a map to them:
Spend the quarter on the file and not on the pages, and you’ll have built a very tidy index of material no machine has a reason to quote.
Book a free 15-minute session See the proof