README

Debounced IndexNow pings for Orbis CMS pages.

Requirements

PHP ^8.3
Laravel ^13.0
orbis-cms/core
A queue worker and a shared cache store (redis, database, ...) that holds the debounce state

Installation

composer require orbis-cms/indexnow
php artisan vendor:publish --tag=orbis-cms-indexnow-config
php artisan migrate

The migration creates indexnow_pings, which remembers the pages that have been submitted (see Catching up). In an application with a database per tenant it has to run for every tenant.

Set the key in .env:

INDEXNOW_KEY=your-key

The key is served as plain text at /{key}.txt, and that URL is sent as keyLocation with every ping, so no file has to be placed in public/. The route is not registered automatically: add it to the application's routes, where its middleware, domain or tenant group are known:

// routes/web.php
Route::indexNowKey();

// or inside a group
Route::domain('{tenant}.example.com')->group(fn () => Route::indexNowKey());

The macro returns a regular route named indexnow.key, so ->middleware() and the other route methods can be chained. The keyLocation sent to IndexNow is built from the origin of the pinged URL and the path of that route, group prefix included (/site/{key}.txt inside Route::prefix('site')); a prefix with parameters of its own cannot be derived and fails the ping. A ping fails with KeyRouteNotRegistered while no route with that name exists.

Key resolver

The key is obtained through OrbisCms\IndexNow\Contracts\KeyResolver, which receives a URL (the page URL when a ping runs, the request URL when the key file is served) and returns the key for it. A URL without a key is an error: the resolver throws KeyNotResolved, the job fails and the key route answers 404. To leave a page or site out of IndexNow, return null from the UrlResolver instead. The default ConfigKeyResolver returns indexnow.key for every URL. For multi-site setups, implement the contract and bind it in the application's service provider:

class SiteKeyResolver implements KeyResolver
{
    public function resolve(string $url): string
    {
        return Site::where('host', parse_url($url, PHP_URL_HOST))->value('indexnow_key')
            ?? throw KeyNotResolved::forUrl($url);
    }
}

// AppServiceProvider::register()
$this->app->bind(KeyResolver::class, SiteKeyResolver::class);

The key route answers with the key only when the requested {key} equals the key resolved for the request URL (behind a proxy, TrustProxies must be configured so that URL is the public one); otherwise it responds with 404. Keys must be 8–128 characters of a-z, A-Z, 0-9 and - (the format of the protocol, which the key route accepts as well); a resolver returning anything else fails the ping with KeyNotResolved, whichever resolver it is.

How it works

Saving a Page, or a pageable model that uses NotifiesIndexNow, resolves the page's URL through UrlResolver (by default Page::url(), so it is built from the saving request) and dispatches PingIndexNowJob carrying the page and that URL. The job is delayed by delay_minutes.
The ping is debounced per URL with the framework's job debounce: every save of the same page dispatches a new job and moves the ping to delay_minutes after that save, and the jobs it supersedes are discarded when they come due. A burst of edits therefore produces one ping, sent after the page has been quiet for the whole window. A changed URL (a renamed slug) produces a separate ping for the new address.
When the job runs, PageEligibility decides whether the page may still be submitted. The default implementation requires that the page:
- exists and is not soft-deleted;
- is published, together with its pageable (including publication schedules);
- is indexable (indexable flag and pageIndexable());
- does not belong to a redirect;
- is not vetoed by its pageable: it either does not implement IndexNowable or indexNowAllowed() returns true.
These are the rules of the default sitemap source except sitemapEnabled(): a page left out of the sitemap may still be submitted, use indexNowAllowed() to prevent it. A page unpublished, made non-indexable or deleted while its ping is pending is not submitted. Unpublishing and deletion are not reported to IndexNow.
When the job runs it submits the URL. HTTP 429 and 5xx responses are retried according to tries and backoff. Other rejections are logged as warnings and not retried.

Page changes are observed automatically. To also ping when a pageable model itself changes, add the trait:

use OrbisCms\IndexNow\Models\NotifiesIndexNow;

class Article extends Model implements Pageable
{
    use HasPage, NotifiesIndexNow;
}

Extension points

The package knows nothing about tenants or sites. Multi-tenant (or any other) behaviour is attached from the application through these points:

Point	Called	Purpose
`Contracts\UrlResolver::resolve(Page): ?string`	on save, in the saving request	The URL to submit; `null` skips the page. Default: `Page::url()`.
`Contracts\KeyResolver::resolve(string $url): string`	in the worker and when the key file is served	The key for a URL; throws `KeyNotResolved` when there is none.
`Contracts\PageEligibility::allows(Page): bool`	in the worker	Whether the page may still be submitted.
`indexnow.job`	on dispatch	Job class. Extend `PingIndexNowJob` to add middleware, a queue-tenancy marker interface, etc.
`Route::indexNowKey()`	in the application's routes	Registers the key file route (`indexnow.key`); place it in any group, with any middleware or domain.

Bind the contracts in the application's service provider. The job carries the Page and the URL, so a queue that restores the application's context before running jobs (tenant-aware queues) makes the page load, the eligibility check and KeyResolver run in that context.

Notes:

The URL is resolved while the model is saved. A save outside a request (a console command, a seeder) yields APP_URL with the default resolver; bind a UrlResolver that knows better.
Behind a proxy the application must trust it (TrustProxies), otherwise request URLs differ from public ones.
A key resolved for a page URL must be the one served for the same URL at /{key}.txt; both go through KeyResolver.

Pinging manually

Only saves of a Page and of models using NotifiesIndexNow are detected. Anything else (a page whose visibility depends on another model, a scheduled command that finds pages that have just become visible, an import that writes without events) is reported by the application:

use OrbisCms\IndexNow\Facades\IndexNow;

// a catalog was published: its children became visible
IndexNow::ping($catalog->children);   // pages or pageable models, one or many

ping() follows the same path as a detected save, minus the changed-attribute check: the URL is resolved by the UrlResolver in the calling context, the ping is debounced per URL, the jobs are dispatched after the surrounding transaction commits, and the PageEligibility decides when the job runs, so passing pages that are not (yet) eligible is harmless. It does nothing while enabled is false.

Call it where the application knows the right context, for example inside the tenant that owns the pages.

Catching up

A ping can be lost or never scheduled: a job fails for good, a page was invisible when it was saved and became visible later, the site existed before the package. The job records every accepted submission in indexnow_pings, and indexnow:sync submits the published, eligible pages that have no record yet:

The package does not register the command or schedule it; the application opts in:

// bootstrap/app.php
->withCommands([OrbisCms\IndexNow\Console\SyncCommand::class])

// routes/console.php
Schedule::command('indexnow:sync')->everyFiveMinutes();

The pings of the command are not debounced; they are sent at once.
At most 500 pages are queued per run; change it with --limit (Schedule::command('indexnow:sync --limit=200')). A large site is therefore submitted gradually, and the rest is picked up by the next runs.
Only pages that were never submitted are considered. A page changed after its first ping is handled by the save observer, and a page that went hidden and visible again without being saved is reported with IndexNow::ping().
A page IndexNow keeps rejecting has no record and is retried by every run, with a warning in the log each time; fixing the cause (usually the key) is the application's job.
The command builds URLs through the UrlResolver without a request. A multi-tenant application runs it once per tenant, inside that tenant, after setting the root URL itself (URL::forceRootUrl()) or with a UrlResolver that knows the tenant.

Opting a model out

A Pageable model can veto pings for its pages by implementing OrbisCms\IndexNow\Contracts\IndexNowable (NotifiesIndexNow provides defaults for both of its methods, so override only what you need; as with other capability interfaces, the trait alone is not enough: a model that uses it without declaring implements IndexNowable throws a LogicException when it boots). The method is evaluated when the ping runs, so a change of its result while a ping is pending is respected:

class Article extends Model implements IndexNowable, Pageable
{
    public function indexNowAllowed(): bool
    {
        return ! $this->is_private;
    }
}

Ignored attributes

A save that changes only ignored attributes does not schedule a ping. NotifiesIndexNow ignores updated_at by default; a pageable replaces the list by overriding indexNowIgnoredAttributes() from IndexNowable:

public function indexNowIgnoredAttributes(): array
{
    return ['updated_at', 'views', 'last_viewed_at'];
}

Page changes ignore updated_at only. Creating a page schedules a ping, since all of its attributes are new.

Configuration

Option	Env	Default	Description
`enabled`	`INDEXNOW_ENABLED`	`true` in production, otherwise `false`	Master switch.
`key`	`INDEXNOW_KEY`	`null`	Key returned by the default resolver. Without a key the job fails with `KeyNotResolved`.
`endpoint`	`INDEXNOW_ENDPOINT`	`https://api.indexnow.org/indexnow`	Submission endpoint.
`delay_minutes`	`INDEXNOW_DELAY_MINUTES`	`15`	Quiet period after the latest save before the ping is sent.
`queue`	`INDEXNOW_QUEUE`	`default`	Queue name.
`connection`	`INDEXNOW_QUEUE_CONNECTION`	`null`	Queue connection.
`job`	–	`PingIndexNowJob`	Job class.
`tries`	–	`3`	Attempts per ping.
`backoff`	–	`[60, 300]`	Seconds between attempts.

Testing

composer test
composer pint:test

orbis-cms / indexnow

Package info

Security

Statistics