README

This TYPO3 extension provides an alternative Markdown representation of your pages for AI bots and crawlers.

This makes content accessible not just for humans and screen readers, but also for AI systems that consume and process web content.

Features

Converts any TYPO3 page to Markdown on-the-fly via content negotiation
Works automatically with your existing page templates - no TypoScript configuration needed
Extracts content from <main> element (or <body> as fallback)
Automatic <link rel="alternate"> tag for Markdown discovery
Strips navigation, header, footer, and other non-content elements
Includes page metadata (title, dates, description, categories) as YAML front matter

Installation

Use composer req b13/ai-bots-love-markdown or install it via TYPO3's Extension Manager.

After installation, add the site set to your site configuration:

dependencies:
  - b13/ai-bots-love-markdown

Configuration

Site Settings

The extension provides two settings that can be configured per site (both enabled by default):

Setting	Default	Description
`ai_bots_love_markdown.enableContentNegotiation`	`true`	Enable content negotiation via `Accept: text/markdown` header
`ai_bots_love_markdown.enableDiscoveryTag`	`true`	Add `<link rel="alternate">` tag to HTML pages for Markdown discovery

To override these settings, add them to your site's settings.yaml:

ai_bots_love_markdown:
  enableContentNegotiation: true
  enableDiscoveryTag: false

Currently, the suffix /ai-bots-love.md is hardcoded on purpose.

Usage

Access Methods

Accept header: Request any page with Accept: text/markdown header
```
curl -H "Accept: text/markdown" https://example.com/my-page/
```
URL suffix: Append /ai-bots-love.md to any page URL
```
https://example.com/ai-bots-love.md
```

Output Format

The extension outputs Markdown with YAML front matter, extracting metadata from HTML meta tags with fallback to TYPO3 page record data:

---
title: "From og:title > <title> > page record"
url: "From canonical URL > request URL"
description: "From og:description > meta description > page record"
image: "From og:image (if present)"
author: "From meta author > page record"
date: 2024-01-15
modified: 2024-01-20
keywords:
  - From meta keywords
categories:
  - From TYPO3 categories
type: "From og:type (if present)"
site: "From og:site_name (if present)"
locale: "From og:locale (if present)"
---

# Page Title

Content converted to Markdown...

Metadata Priority:

title: og:title → <title> tag → page record
url: canonical link → request URL
description: og:description → meta description → page record
image: og:image (if present)
author: meta author → page record
date/modified: page record (crdate/tstamp)
keywords: meta keywords (if present)
categories: TYPO3 sys_category (from database)

Auto-Discovery

The extension automatically adds a <link> tag to all HTML pages for Markdown discovery:

<link rel="alternate" type="text/markdown" href="https://example.com/my-page/ai-bots-love.md" title="Markdown version" />

Opt-in Behavior

Pages are only converted to Markdown if they contain the <link rel="alternate" type="text/markdown"> tag. This means:

Pages without the site set enabled won't be converted
You can disable conversion for specific pages by disabling the discovery tag

Technical Details

How It Works

A PSR-15 middleware intercepts requests with Accept: text/markdown header or /ai-bots-love.md suffix
The normal page rendering proceeds (your existing templates, TypoScript, etc.)
The middleware checks if the response contains the markdown alternate link tag
If present, it extracts <main> content (or <body> as fallback)
The HTML is converted to Markdown using league/html-to-markdown
Navigation, header, footer, and other non-content elements are stripped
Page metadata is added as YAML front matter

Response Headers

The Markdown response includes:

Content-Type: text/markdown; charset=utf-8
X-Robots-Tag: noindex (to prevent indexing of Markdown version)

Best Practices for Templates

For best results, wrap your main content in a <main> element:

<body>
    <header>...</header>
    <nav>...</nav>
    <main>
        <!-- This content will be converted to Markdown -->
        <f:render section="Main" />
    </main>
    <footer>...</footer>
</body>

License

The extension is licensed under GPL v2+, same as the TYPO3 Core.

Background & Authors

This extension was created by b13 GmbH in 2026 to enable AI systems to better consume website content. As AI crawlers become increasingly important for content discovery and processing, providing a clean Markdown representation helps ensure your content is accurately understood and indexed by AI systems.

Huge credits to Dries Buytaert's inspiration on this topic:

Find more TYPO3 extensions we have developed that help us deliver value in client projects. As part of the way we work, we focus on testing and best practices to ensure long-term performance, reliability, and results in all our code.

b13 / ai-bots-love-markdown

Maintainers

Details