llms.txt: The Complete Guide for AI Search & LLM Discovery (2026)
By the Maybe Labs teamllms.txt is a plain-text file at your site root — `/llms.txt` — that lists your most important pages in a format large language models can read quickly. Think of it as a curated map for AI crawlers: what your product is, which pages answer which questions, and where to send someone who asks 'What's the best tool for X?' It's a core tactic in Answer Engine Optimization (AEO) and Generative Engine Optimization (GEO).
What llms.txt is (and isn't)
- Is: a human-readable index of key URLs with short descriptions.
- Is: a signal to AI systems about which pages you want cited.
- Isn't: a replacement for sitemap.xml — sitemaps list everything; llms.txt curates the best.
- Isn't: a robots.txt directive — it doesn't block or allow crawlers.
- Isn't: guaranteed ranking — it helps discovery and citation, not magic placement.
Maybe Labs publishes llms.txt alongside robots.txt that explicitly allows AI crawlers (GPTBot, Google-Extended, PerplexityBot).
How to write an llms.txt file
- Start with a one-line product summary in a blockquote (`> …`).
- Group links under clear headings: Core pages, Use cases, Key guides, Comparisons.
- Use markdown links: `Title`.
- Add a Notes section for disambiguation ('We make X, not Y').
- Keep it under ~100 links — curation beats comprehensiveness.
- Update when you publish definitive new guides.
llms.txt + AEO: the full stack
| Tactic | Purpose |
|---|---|
| llms.txt | Curated page index for LLMs |
| FAQ schema on guides | Structured Q&A for citations |
| Article JSON-LD | Entity + freshness signals |
| robots.txt AI allowances | Ensure crawlers can read content |
| RSS + IndexNow | Fast discovery of new posts |
| Definitive long-form guides | Quotable, comprehensive answers |
For the full playbook see answer engine optimization guide and generative engine optimization for startups.
llms.txt FAQ
What is llms.txt?
A plain-text file at /llms.txt that lists your site's most important pages with descriptions, helping AI systems and LLMs discover and cite your content.
Do I need llms.txt if I have a sitemap?
Yes — they serve different purposes. Sitemaps list all indexable URLs; llms.txt curates the pages you want AI models to prioritize and understand.
Does llms.txt improve SEO?
It primarily helps AI search and LLM citation (AEO/GEO). Traditional SEO still relies on sitemaps, content quality, and backlinks — but AEO is increasingly where discovery happens.
Where should llms.txt live?
At your site root: https://yourdomain.com/llms.txt — same level as robots.txt.
If you want AI search to cite your startup, start with one definitive guide, add FAQ schema, allow AI crawlers, and publish llms.txt. Maybe Labs practices what we preach — read ours at maybelabs.ai/llms.txt.
Make your next launch in motion
Maybe Labs turns prompts into product launch and update videos: story, assets, and final cut, start to end.
Get early access →