darvinstudio/darvin-crawler-bundle

This bundle provides console command that detects broken links on your website.

6.0.4 2021-04-19 05:59 UTC

Requires

• symfony/console: *
• symfony/css-selector: *
• symfony/dom-crawler: *
• symfony/http-client: *

MIT

• Darvin Studio

README

This bundle provides console command that detects broken links on your website.

Sample configuration

# config/packages/dev/darvin_crawler.yaml
darvin_crawler:
    default_uri: https://example.com # Default value of command's "uri" argument
    blacklists:
        parse: # Content from URIs matching these regexes will not be parsed
            - '/\/filtered\//'
        visit: # URIs matching these regexes will not be visited
            - '/\/filtered\//'

Usage

Crawl default URI:

$ bin/console darvin:crawler:crawl

Crawl specified URI:

$ bin/console darvin:crawler:crawl https://example.com

Display all visited links:

$ bin/console darvin:crawler:crawl https://example.com -v