README

🔗 Demo: yu.net.ua/llms.txt · yu.net.ua/llms-full.txt

Yu_LlmsTxt serves two files from the root of every store view:

/llms.txt - a short Markdown "map" of the store: what it sells, its main categories, its key pages (shipping, returns, about), and optionally a handful of featured products;
/llms-full.txt - the same map plus the full text of those pages, so an AI assistant can read the store's rules without following a single link.

Both are built automatically from your catalog and CMS content, kept up to date when you edit them, and configured per store view.

Requirements

PHP >= 8.1
Magento 2.4.x

Installation

composer require yu-dev/module-llms-txt
bin/magento module:enable Yu_LlmsTxt
bin/magento setup:upgrade
bin/magento cache:enable yu_llms_txt
bin/magento cache:flush

cache:enable switches on the module's own cache type (LLMs.txt in System → Cache Management): Magento leaves cache types added by a module disabled until you enable them. Without it the files still work, but are rebuilt on every request.

The module is enabled by default; open https://your-store/llms.txt to see the result, and fine-tune it under Stores → Configuration → Catalog → LLMs.txt.

If your web server serves existing files directly, make sure there is no physical llms.txt in the document root - it would take precedence over the generated one.

What llms.txt is, and why a store wants one

Search engines have robots.txt and sitemap.xml. AI assistants (ChatGPT, Claude, Perplexity, shopping agents, coding tools) have a newer convention: llms.txt. It is a plain Markdown file at the root of a site that answers two questions an AI reader has when it arrives: what is this site? and where is the important content?

The difference from a sitemap matters. A sitemap lists every URL for a crawler to walk through. An AI assistant works differently: it reads a file whole, into a limited "working memory" (its context window), and answers from it. So llms.txt is deliberately short and curated - a table of contents with one-line descriptions, not a dump of everything.

For a store, the payoff is simple: when a shopper asks an AI assistant "does this shop ship to Canada?" or "what's their return policy?", the assistant finds the answer in your own words instead of guessing.

What goes into the file

# Example Store

> Outdoor clothing and gear for hiking and camping. Ships worldwide.

(your own Markdown: contacts, shipping regions, key rules...)

## Categories
- [Dept A](https://example.com/dept-a.html): Short category description
  - [Tops](https://example.com/dept-a/tops.html)
- [Dept B](https://example.com/dept-b.html)

## Pages
- [Shipping Policy](https://example.com/shipping): How and where we ship
- [Returns](https://example.com/returns)

## Featured Products
- [Trail Jacket](https://example.com/trail-jacket.html): Short product description

## Optional
- [Sitemap](https://example.com/sitemap.xml): Full list of catalog and content pages

Section by section:

Header - the title (your store name unless you set another), a one- or two-sentence summary shown as a quote, then any Markdown you want to add: contacts, delivery regions, the rules you most want an assistant to get right.
Categories - the storefront menu as a nested list: active categories shown in the menu, down to the depth you choose. The description comes from the category's meta description, or the start of its description.
Pages - the CMS pages you pick. If you pick none, every active page of the store view is listed except the ones Magento itself uses as the home page, the 404 page and the "cookies disabled" page.
Featured Products - off by default; see below.
Optional - a link to your XML sitemap, once one has been generated. ("Optional" is the llms.txt name for links an assistant may skip.)

In llms-full.txt, each listed CMS page is followed by its full text, and each category by its full description. Page content is converted from HTML (including Page Builder layouts) to clean Markdown: headings, lists, links and tables are kept, everything decorative is dropped, and widgets and {{store url}}-style directives are resolved first.

Why products are a short list, never the whole catalog

It is tempting to list every product. It is a bad idea:

Size. Thousands of product lines push the important part - your policies - out of the assistant's working memory.
Staleness. Prices and stock change constantly; a cached text file that says "in stock, $45" is worse than no answer. That is also why the file contains no prices at all - the product page stays the source of truth.
Duplication. The full list already exists in your XML sitemap and in the structured data on product pages.

So the product section is a short curated list - your flagship products or bestsellers - capped at a limit you set (20 by default).

Configuration

Stores → Configuration → Catalog → LLMs.txt (every setting can be changed per store view).

General

Setting	Default	What it does
Enabled	Yes	Turns both files on for the store view. While off, both URLs return a normal 404.
Title	(empty)	The file's heading. Empty = the store name.
Summary	(empty)	One or two sentences under the title.
Custom Markdown	(empty)	Inserted as-is after the summary.
Generate llms-full.txt	Yes	Serves `/llms-full.txt` as well.

Categories - Include Categories (Yes), Max Depth (2; 1 = top-level categories only).

CMS Pages - Pages: pick the pages to list. Nothing picked = all active pages except home, 404 and "cookies disabled".

Featured Products

Setting	Default	What it does
Include Products	No	Adds the Featured Products section.
Source	Marked products	Marked products, Bestsellers, or Marked products, then bestsellers.
Limit	20	Maximum number of products.
Bestsellers Period, Months	3	How far back bestsellers are counted.

Only products a shopper can actually open and buy are listed: enabled, visible in the catalog, in stock, assigned to the store's website.

Marking products

The module adds a product attribute, Include in llms.txt (Yes/No, per store view), in the Search Engine Optimization group of the product form. To mark many products at once, select them in the product grid and use Actions → Update Attributes.

Bestsellers

Bestsellers come from Magento's own sales report data, which Magento aggregates in the background. Until it has been built at least once (Reports → Statistics → Refresh Statistics, or its cron job), the bestseller list is empty. When a bestseller is a variant of a configurable (or grouped/bundle) product, its parent product is listed, since that is the page a shopper can open.

Keeping it up to date

Generated files are stored in a cache type of their own, LLMs.txt (System → Cache Management), one entry per store view and file. It is cleared automatically when:

a category, CMS page or product is saved or deleted, or a category is moved;
products are changed in bulk (Update Attributes, website assignment);
a sitemap is generated;
the module's settings are saved.

Entries also expire after 24 hours, which covers the one thing that changes without any save: bestseller statistics.

Extending: adding your own section

Each part of the file is a separate section provider - a class implementing Yu\LlmsTxt\Api\SectionProviderInterface:

public function getSection(int $storeId, bool $full): string;

It returns a Markdown fragment (or '' to leave the section out); $full is true when building llms-full.txt. Register it from your own module's etc/di.xml; sortOrder decides where it appears (the built-in sections use 10, 20, 30, 40 and 100):

<type name="Yu\LlmsTxt\Model\Generator">
    <arguments>
        <argument name="sections" xsi:type="array">
            <item name="faq" xsi:type="array">
                <item name="provider" xsi:type="object">Vendor\Module\Model\FaqSection</item>
                <item name="sortOrder" xsi:type="number">35</item>
            </item>
        </argument>
    </arguments>
</type>

A section that throws an exception is logged and skipped - the rest of the file is still served.

Glossary

llms.txt - a Markdown file at the root of a website that tells AI assistants what the site is and where its important content lives (proposed at llmstxt.org).
llms-full.txt - the same, with the full text of the listed pages included, so the assistant does not need to open each link.
Context window - the amount of text an AI model can read at once; the reason llms.txt must stay short.
Store view - Magento's unit for a language/storefront; each one gets its own files and settings.

Author

Yuriy Akishin:

📧 Email: yuriy.akishin@gmail.com
💼 LinkedIn: https://www.linkedin.com/in/yuriyakishin/
💻 GitHub: https://github.com/yuriyakishin

License

MIT

yu-dev / module-llms-txt

Package info

Security

Statistics