omnicrawl / zyte

1.x-dev 2026-11-10 07:18 UTC

Requires

• php: >=8.2
• omnisense/contracts: ^1.0@dev
• omnisense/omnicrawl: ^1.0@dev
• symfony/http-client: ^6.4|^7.0|^8.0

Requires (Dev)

• phpstan/phpstan: ^2.1
• phpunit/phpunit: ^11.0

MIT

• Marco Meyer

• scraping
• responsible
• framework-agnostic
• zyte
• omnicrawl
• zyte-api

Published on Packagist.org on 2026-11-10 08:08 UTC

README

Zyte API for omnisense/omnicrawl, under any PHP host. Pages over HTTP (POST /v1/extract) with the target's headers - for targets declared as one's own site or with written permission only, because Zyte's ban avoidance cannot be turned off.

It is reached through omnicrawl's gate only: every address is admitted first (the target's declaration, robots.txt read by the core, TDM reservations, the host's limits), the service is called after, and what it brings back is checked again and made a record with its provenance. Read what omnicrawl refuses to do, and why.

$registry = new Registry([new ZyteFactory($http)], [
    'zyte' => ['factory' => 'zyte', 'options' => ['api_key' => 'ZYTE_API_KEY']],
], $gate);

The key goes by reference: ZYTE_API_KEY names an environment variable (or a secret of the SecretsInterface given to the factory), never the key itself. Its usage policy is policy.json (limits, terms, what they say of lawful use), read on the day it says. See docs/index.md.

Tested on recorded answers written from the service's documentation; no test calls the service.

MIT licence.