tomasnorre/crawler

Crawler extension for TYPO3

12.0.10 2025-10-28 14:37 UTC

Requires

• php: ~8.1.0 || ~8.2.0 || ~8.3.0 || ~8.4.0
• ext-json: *
• ext-pdo: *
• ext-posix: *
• beberlei/assert: ^3.3
• doctrine/dbal: ^3.10 || ^4.3
• guzzlehttp/guzzle: ^6.4.1 || ^7.2
• psr/http-message: ^2.0
• psr/http-server-handler: ^1.0
• psr/http-server-middleware: ^1.0
• psr/log: ^1.0 || ^2.0 || ^3.0
• symfony/console: ^6.4 || ^7.0
• symfony/service-contracts: ^3.6
• typo3/cms-backend: ^12.4 || ^13.4
• typo3/cms-core: ^12.4 || ^13.4
• typo3/cms-extbase: ^12.4 || ^13.4
• typo3/cms-frontend: ^12.4 || ^13.4
• typo3/cms-info: ^12.4 || ^13.4
• typo3/cms-seo: ^12.4 || ^13.4

Requires (Dev)

• codeception/codeception: ^5.0
• codeception/module-asserts: ^3.0
• codeception/module-filesystem: ^3.0
• codeception/module-phpbrowser: ^3.0
• codeception/module-webdriver: ^4.0
• infection/infection: ^0.29.9
• nikic/php-parser: ^4.13 || ^5.3
• phpspec/prophecy-phpunit: ^2.0
• phpstan/extension-installer: ^1.2
• phpstan/phpstan: ^2.0
• phpstan/phpstan-deprecation-rules: ^2.0
• phpunit/phpunit: ^10.5 || ^11.4
• roave/security-advisories: dev-latest
• sbuerk/typo3-cmscomposerinstallers-testingframework-bridge: ^0.1
• shipmonk/composer-dependency-analyser: ^1.8
• ssch/typo3-rector: ^3.0
• symfony/yaml: ^6.4 || ^7.0
• symplify/easy-coding-standard: ^12.2
• symplify/phpstan-rules: ^14.2
• tomasvotruba/cognitive-complexity: ^1.0
• tomasvotruba/type-coverage: ^2.0
• typo3/testing-framework: ^8.0 || ^9.0

Suggests

• typo3/cms-seo: Enables the posibility to priorities your Crawler Queue + You have seo features in the CMS it self.

Replaces

• aoepeople/crawler: 12.0.10
• typo3-ter/crawler: 12.0.10

GPL-3.0

• Tomas Norre Mikkelsen

README

TYPO3 Crawler crawls the TYPO3 page tree. Used for cache warmup, indexing, publishing applications etc.

You can include the crawler in your TYPO3 project with composer or from the TYPO3 Extension Repository

composer require tomasnorre/crawler

Crawler processes

Versions and Support

Release	TYPO3	PHP	Fixes will contain
12.x.y	12.4-13.3	8.1-8.4	Features, Bugfixes, Security Updates, Since 12.0.6 TYPO3 13.4, Since 12.0.7 PHP 8.4
11.x.y	10.4-11.5	7.4-8.1	Security Updates, Since 11.0.3 PHP 8.1
10.x.y	9.5-11.0	7.2-7.4	Security Updates
9.x.y	9.5-11.0	7.2-7.4	As this version has same requirements as 10.x.y, there will be no further releases of this version, please update instead.
8.x.y			Releases do not exist
7.x.y			Releases do not exist
6.x.y	7.6-8.7	5.6-7.3	Security Updates

Documentation

Please read the documentation

To render the documentation locally, please use the official TYPO3 Documentation rendering Docker Tool. https://github.com/t3docs/docker-render-documentation

Contributions

Please see CONTRIBUTING.md

Honorable Previous Maintainers

• Kasper Skaarhoj
• Daniel Poetzinger
• Fabrizio Branca
• Tolleiv Nietsch
• Timo Schmidt
• Michael Klapper
• Stefan Rotsch