lastcall/last-call-crawler

Static site crawler/processor

1.1.1 2015-12-31 21:27 UTC

Requires

• php: >=5.5.0
• guzzlehttp/guzzle: ^6.1
• pimple/pimple: ^3.0
• psr/log: ^1.0
• symfony/console: ~2.8|~3.0
• symfony/dom-crawler: ~2.8|~3.0
• symfony/event-dispatcher: ~2.8|~3.0
• symfony/filesystem: ~2.8|~3.0

Requires (Dev)

• doctrine/dbal: ^2.5
• fabpot/php-cs-fixer: ^1.11
• phpunit/phpunit: ^4.8
• symfony/css-selector: ~2.8|~3.0
• symfony/stopwatch: ~2.8|~3.0

MIT

• Rob Bayliss

README

This is a CLI tool to crawl a website and do processing on the data that's found. The most common use case would be for scraping content off of an HTML site, but it could also be used to invoke an API and process the data.

Installation

Install the crawler using Composer.

    php composer.phar require lastcall/last-call-crawler

You can also add the dependency directly to your project's composer.json:

    {
        "require": {
            "lastcall/last-call-crawler": "~1.0"
        }
    }

Documentation can be found here:

• Configurations
• Running
• Handlers