schliesser/sitecrawler

TYPO3 sitemap crawler

Installs: 2 398

Dependents: 0

Suggesters: 0

Security: 0

Stars: 4

Watchers: 1

Forks: 1

Open Issues: 2

Type:typo3-cms-extension

v1.0.1 2020-08-12 09:03 UTC

This package is auto-updated.

Last update: 2021-03-17 08:48:31 UTC


README

This extension provides a Symfony command to crawl the sitemap including all sub sitemaps. It gathers all available urls and then calls each url to warm up the TYPO3 page cache.

You can provide custom request headers in json format.

Examples

Composer based:

vendor/bin/typo3cms sitecrawler:crawl 'https://www.example.com/sitemap.xml'

# with custom request headers
vendor/bin/typo3cms sitecrawler:crawl 'https://www.example.com/sitemap.xml' '{"X-Pjax": true}'

Non composer:

typo3/sysext/core/bin/typo3 sitecrawler:crawl 'https://www.example.com/sitemap.xml'