repat/spatie-crawler-redis

Redis CrawlQueue for spatie/crawler

0.1.1 2019-09-05 17:27 UTC

README

Latest Version on Packagist Total Downloads

spatie-crawler-redis is an alternative CrawlerQueue implementing the Spatie\Crawler\CrawlQueue\CrawlQueue interface using Redis Hashes.

Installation

$ composer require repat/spatie-crawler-redis

Example

Create a Predis\Client beforehand if you need options, such as selecting a database. If you don't pass a client, a new one without options will be used. Predis assumes 127.0.0.1, 6379 and 0 as default host, port and database. You can also pass a custom prefix, otherwise uniqid() will be used.

use Repat\CrawlQueue\RedisCrawlQueue;

// see https://github.com/nrk/predis for options
$options = [
    'database' => 7,
];

$prefix = uniqid() . ':'; // same as passing no prefix

$redisClient = new \Predis\Client($options);

// ...
->setCrawlQueue(new RedisCrawlQueue($redisClient, $prefix))

// uses new \Predis\Client without options
->setCrawlQueue(new RedisCrawlQueue())

TODO

  • phpredis support

Testing

Thanks spatie for the tests. These are the instructions:

To run the tests you'll have to start the included node based server first in a separate terminal window.

cd tests/server
npm install
./start_server.sh

With the server running, you can start testing.

vendor/bin/phpunit

License

Version

  • Version 0.1.1

Contact

repat

Flattr this git repo