umpirsky/centipede-crawler

Crawls all unique links.

0.1.2 2016-06-13 10:09 UTC

Requires

• php: >=5.4
• guzzlehttp/guzzle: ~5.3

Requires (Dev)

• phpspec/phpspec: ~2.0

MIT

• Saša Stamenković

README

symfony upgrade fixer • twig gettext extractor • wisdom • centipede • permissions handler • extraload • gravatar • locurro • country list • transliterator

Centipede Crawler

Crawls all unique links.

Usage

$urls = (new Centipede\Crawler('http://dev.umpirsky.com'))->crawl();

Asynchronous

(new Centipede\Crawler('http://dev.umpirsky.com'))->crawl(function ($url, GuzzleHttp\Message\FutureResponse $response) {
    printf('(%d) %s', $response->getStatusCode(), $url);
});