README

Acorn LLMs.txt

Expose your WordPress site content through /llms.txt endpoints following the llmstxt.org specification. This makes your WordPress content easily accessible to Large Language Models (LLMs) in a structured, markdown format.

Features

llmstxt.org compliant: Implements the standard specification for LLM-readable content
Multiple endpoints: /llms.txt (listing), /llms-full.txt (complete content), /llms-small.txt (excerpts), individual post endpoints
XML sitemap integration: /llms-sitemap.xml with automatic SEO plugin integration (Yoast, RankMath, The SEO Framework, WordPress core)
SEO integration: Respects noindex settings and adds X-Robots-Tag headers to prevent search engine indexing
WooCommerce support: Product SKUs, prices, and metadata automatically included
Hierarchical pages: Displays page hierarchy with proper indentation
High performance: Built-in Laravel Cache integration with automatic invalidation
WordPress shortcode processing: Converts shortcodes to readable content before markdown conversion
Extensible: Developer hooks and filters for customization
Configurable: Comprehensive configuration options for content filtering and formatting

Installation

Install via Composer:

composer require roots/acorn-llms-txt

Publish the configuration file:

wp acorn vendor:publish --provider="Roots\AcornLlmsTxt\Providers\LlmsTxtServiceProvider"

Endpoints

`/llms.txt`

A structured listing of your site content (like a table of contents):

# Your Site Name

> Your site tagline or description (if available)

## Additional Resources

- [Full Content](https://yoursite.com/llms-full.txt) - Complete content of all posts and pages in Markdown format

## Recent Posts

- [Hello World (Post | Category: Uncategorized)](https://yoursite.com/hello-world/) - Welcome to WordPress. This is your first post.

## Pages

- [About (Page)](https://yoursite.com/about/) - Information about your site
  - [Contact (Page)](https://yoursite.com/about/contact/) - Get in touch with us

`/llms-full.txt`

Complete content of all posts and pages in markdown format:

# Your Site Name

---

## Hello World
URL: https://yoursite.com/hello-world/
Date: 2025-01-01
Author: Admin
Type: Post
Category: Uncategorized

Welcome to WordPress. This is your first post. Edit or delete it, then start writing!

`/llms-small.txt`

Compressed version with excerpts only for faster processing:

# Your Site Name (Excerpts)

## Hello World (Post | Category: Uncategorized)
URL: https://yoursite.com/hello-world/
Date: 2025-01-01
Author: Admin

Welcome to WordPress. This is your first post...

`/llms-sitemap.xml`

XML sitemap listing all llms.txt endpoints, automatically integrated with SEO plugins.

Individual Post Endpoints (Optional)

Access individual posts via /post-slug.txt (disabled by default for performance).

Configuration

After publishing, edit config/llms-txt.php:

Post Types

'post_types' => ['post', 'page'],

Content Processing

'content' => [
    'process_shortcodes' => true, // Process WordPress shortcodes
    'include_featured_image' => false,
    'include_author' => true,
    'include_date' => true,
    'include_taxonomies' => true,
    'excerpt_length' => 20,
    'date_format' => 'Y-m-d',
],

Content Limits

'limits' => [
    'max_posts' => 1000,
    'max_content_length' => 50000,
    'posts_per_section' => 50,
],

Exclusions

'exclude' => [
    'post_ids' => [123, 456], // Specific posts to exclude
    'password_protected' => true,
    'sticky_posts' => false,
],

Individual Post Endpoints

'individual_posts' => [
    'enabled' => false, // Enable /post-slug.txt endpoints
    'post_types' => ['post', 'page'],
    'cache_ttl' => 3600,
],

Caching

'cache_ttl' => 3600, // 1 hour in seconds

WooCommerce Integration

'woocommerce' => [
    'enabled' => true, // Enable WooCommerce integration
    'include_price' => true, // Include product price
    'include_stock_status' => false, // Include stock status
    'include_product_type' => false, // Include product type
],

SEO Integration

The package automatically respects SEO plugin settings and integrates with their sitemaps:

Content Filtering:

Yoast SEO: Excludes posts/pages with noindex meta
RankMath: Excludes posts/pages with noindex settings
The SEO Framework: Excludes posts/pages marked as noindex

Sitemap Integration:

Yoast SEO: Adds /llms-sitemap.xml to main sitemap index
RankMath: Registers llms sitemap in sitemap array
The SEO Framework: Creates custom sitemap endpoint
WordPress Core: Integrates with wp_sitemaps system

Search Engine Protection:

Adds X-Robots-Tag: noindex header to all llms.txt endpoints to prevent search engine indexing

Posts marked as noindex by SEO plugins will not appear in any llms.txt endpoints.

WP-CLI Commands

View package information

wp acorn llms-txt

Clear cached content

wp acorn llms-txt clear-cache

Developer Hooks

Filters

Filter which post types are included:

add_filter('acorn/llms_txt/post_types', function ($postTypes) {
    return array_merge($postTypes, ['custom_post_type']);
});

Exclude specific posts:

add_filter('acorn/llms_txt/exclude_post', function ($exclude, $post) {
    return $post->post_title === 'Secret Post';
}, 10, 2);

Filter individual post data:

add_filter('acorn/llms_txt/post_data', function ($data, $post) {
    $data['custom_field'] = get_post_meta($post->ID, 'custom_field', true);
    return $data;
}, 10, 2);

Filter the complete posts collection:

add_filter('acorn/llms_txt/posts', function ($posts) {
    return $posts->sortBy('title');
});

Filter the entire document before output:

add_filter('acorn/llms_txt/document', function ($document) {
    // Modify the LlmsTxtDocument object
    return $document;
});

Filter final content before serving:

add_filter('acorn/llms_txt/content', function ($content) {
    return $content . "\n\n<!-- Generated at " . date('Y-m-d H:i:s') . " -->";
});

Filter XML sitemap before serving:

add_filter('acorn/llms_txt/sitemap', function ($sitemap) {
    // Modify the XML sitemap content
    return $sitemap;
});

Control sitemap integration:

add_filter('acorn/llms_txt/include_in_sitemaps', function ($include) {
    // Disable sitemap integration conditionally
    return false;
});

Actions

Add custom sections before default content:

add_action('acorn/llms_txt/before_sections', function ($document) {
    $section = (new \Roots\AcornLlmsTxt\Data\Section)
        ->name('Custom Section')
        ->addLink(
            (new \Roots\AcornLlmsTxt\Data\Link)
                ->title('Custom Link')
                ->url('https://example.com')
                ->details('Custom description')
        );

    $document->addSection($section);
});

Add custom sections after default content:

add_action('acorn/llms_txt/after_sections', function ($document) {
    // Add sections after default content
});

Add completely custom sections:

add_action('acorn/llms_txt/custom_sections', function ($document) {
    // Add your own sections
});

Cache Management

The package automatically invalidates cache when:

Posts are created, updated, or deleted
Post status changes
Post metadata is updated

Manual cache clearing:

wp acorn llms-txt clear-cache

Performance

This package is optimized for performance with:

Efficient caching: Laravel Cache integration reduces database queries
Automatic cache invalidation: Updates cache only when content changes
Memory optimization: Designed to handle large sites with thousands of posts
Configurable limits: Control content length and post counts to manage resource usage

Troubleshooting

Cache Issues

If content isn't updating, clear the cache:

wp acorn llms-txt clear-cache

Routes Not Working

Ensure Acorn routes are optimized:

wp acorn optimize

Performance on Large Sites

For sites with many posts, adjust limits in config/llms-txt.php:

'limits' => [
    'max_posts' => 500,
    'max_content_length' => 25000,
],

Requirements

PHP 8.2+
WordPress with Acorn

Support

GitHub Issues: https://github.com/roots/acorn-llms-txt/issues
Roots Discourse: https://discourse.roots.io/

roots / acorn-llms-txt

Maintainers

Details