README

A Drupal module that provides machine-optimized content browsing API endpoints for AI agents. Companion module to Commerce Agent Detector.

What It Does

AI agents visiting a Drupal site get themed HTML — full of navigation chrome, scripts, and layout markup that obscures the actual content. This module gives agents clean, structured JSON for every page on the site.

The key concept is the resolve endpoint: give it any internal URL and it returns the content as structured data, with all internal links rewritten as API URLs the agent can follow. Agents can browse the site naturally — following links, navigating menus, searching content — without ever parsing HTML.

Requirements

PHP 8.2+
Drupal 10.3+ or 11.x
Commerce Agent Detector ^1.0

Installation

composer require drupal-mods/content-agent-detector
drush en content_agent_detector

API Endpoints

Discovery (Public)

Endpoint	Description
`GET /.well-known/agent-content.json`	Discovery document with all endpoints
`GET /api/content/agent/openapi.json`	Full OpenAPI 3.1.0 specification

Browsing (Authenticated)

Endpoint	Description
`GET /api/content/agent/resolve?path=/about-us`	Resolve any URL to structured content
`GET /api/content/agent/node/{id}`	Get content by node ID
`GET /api/content/agent/search?q=keyword`	Full-text content search
`GET /api/content/agent/list/{type}`	List content by type
`GET /api/content/agent/types`	List available content types
`GET /api/content/agent/menus`	List available menus
`GET /api/content/agent/menu/{name}`	Get menu tree with API links
`GET /api/content/agent/sitemap.json`	All published pages as API links
`GET /api/content/agent/taxonomy/{vocabulary}`	Browse taxonomy terms
`GET /api/content/agent/taxonomy/{vocabulary}/{term}`	Term detail with tagged content
`GET /api/content/agent/breadcrumb?path=/about-us`	Breadcrumb navigation chain

How Agents Browse

Discover — Fetch /.well-known/agent-content.json to find all endpoints
Orient — Get the sitemap or menu tree to see what content exists
Navigate — Use the resolve endpoint to read any page
Follow links — Internal links in responses are rewritten as API URLs
Search — Use the search endpoint for keyword queries

Example: Resolve a Page

GET /api/content/agent/resolve?path=/about-us
Authorization: Bearer cad_your_token_here
Accept: application/json

{
  "status": "success",
  "content": {
    "@type": "WebPage",
    "id": 42,
    "type": "page",
    "title": "About Us",
    "url": "https://example.com/about-us",
    "api_url": "/api/content/agent/node/42",
    "body": "## Our Story\n\nWe are a company that...",
    "fields": {
      "field_subtitle": "Learn more about our team"
    },
    "links": [
      {
        "text": "Meet the Team",
        "url": "/api/content/agent/resolve?path=/team",
        "type": "internal"
      }
    ],
    "taxonomy": {
      "tags": [
        {
          "id": 5,
          "name": "Company",
          "api_url": "/api/content/agent/taxonomy/tags/5"
        }
      ]
    }
  }
}

Example: Menu Navigation

GET /api/content/agent/menu/main

{
  "status": "success",
  "menu": "main",
  "tree": [
    {
      "title": "Home",
      "api_url": "/api/content/agent/resolve?path=/",
      "weight": 0,
      "children": []
    },
    {
      "title": "Products",
      "api_url": "/api/content/agent/resolve?path=/products",
      "weight": 1,
      "children": [
        {
          "title": "New Arrivals",
          "api_url": "/api/content/agent/resolve?path=/products/new",
          "weight": 0,
          "children": []
        }
      ]
    }
  ]
}

Authentication

Uses the same authentication mechanisms as Commerce Agent Detector:

Managed tokens — Created in the admin UI with content scope
Bearer tokens — Authorization: Bearer cad_...
X-Agent-Token header — X-Agent-Token: cad_...
Legacy config tokens — From Commerce Agent Detector settings
Agent detection — Recognized User-Agents (GPTBot, ClaudeBot, etc.)
Anonymous readonly — Optional, for public content sites

Configuration

Navigate to Commerce → Configuration → Agent Detector → Content API tab.

Settings include:

Allowed content types — Which node types to expose
Allowed vocabularies — Which taxonomies to expose
Body format — Markdown (default) or plain text
Max body length — Truncation limit for body content
Include media — Expose images and file attachments
Link rewriting — Rewrite internal links as API URLs
Anonymous readonly — Allow unauthenticated JSON requests

Content Rendering

The module converts Drupal content to clean structured data:

Body fields — HTML converted to Markdown with heading structure, links, lists, code blocks, and tables preserved
Entity references — Resolved to labels with API URLs
Images — Direct file URLs with alt text and dimensions
Files — Download URLs with MIME type and size
Taxonomy — Grouped by vocabulary with API URLs
Media entities — Resolved to source file URLs
Internal links — Rewritten from paths like /about-us to API URLs like /api/content/agent/resolve?path=/about-us

Architecture

This module is a companion to Commerce Agent Detector and reuses its infrastructure:

Service	Provided By
Agent detection	`commerce_agent_detector.agent_detector`
Token management	`commerce_agent_detector.token_manager`
Rate limiting	`commerce_agent_detector.rate_limiter`
Interaction logging	`commerce_agent_detector.logger`
Content rendering	`content_agent_detector.content_renderer`
Link rewriting	`content_agent_detector.link_rewriter`
Access checking	`content_agent_detector.access_check`

Agent interactions with content endpoints appear on the same Commerce Agent Detector dashboard with content_ action prefixes.

Development

# Install dependencies
composer install

# Run unit tests
vendor/bin/phpunit --configuration phpunit.xml --testdox

# Check coding standards
vendor/bin/phpcs --warning-severity=0

# Run static analysis
vendor/bin/phpstan analyse --no-progress

License

GPL-2.0-or-later

drupal-mods / content-agent-detector

Maintainers

Package info

Statistics

Security