omnicrawl / apify

1.x-dev 2026-11-10 07:18 UTC

Requires

• php: >=8.2
• omnisense/contracts: ^1.0@dev
• omnisense/omnicrawl: ^1.0@dev
• symfony/http-client: ^6.4|^7.0|^8.0

Requires (Dev)

• phpstan/phpstan: ^2.1
• phpunit/phpunit: ^11.0

MIT

• Marco Meyer

• scraping
• responsible
• apify
• framework-agnostic
• omnicrawl
• cheerio-scraper

Published on Packagist.org on 2026-11-10 08:12 UTC

README

Apify for omnisense/omnicrawl, under any PHP host. Runs over a fixed set of addresses with the Cheerio Scraper actor: no link followed, one page at a time, no retry, no Apify Proxy, the target's raw HTML, status and headers kept.

It is reached through omnicrawl's gate only: every address is admitted first (the target's declaration, robots.txt read by the core, TDM reservations, the host's limits), the service is called after, and what it brings back is checked again and made a record with its provenance. Read what omnicrawl refuses to do, and why.

$registry = new Registry([new ApifyFactory($http)], [
    'apify' => ['factory' => 'apify', 'options' => ['api_key' => 'APIFY_TOKEN']],
], $gate);

The key goes by reference: APIFY_TOKEN names an environment variable (or a secret of the SecretsInterface given to the factory), never the key itself. Its usage policy is policy.json (limits, terms, what they say of lawful use), read on the day it says. See docs/index.md.

Tested on recorded answers written from the service's documentation; no test calls the service.

MIT licence.