iggi/crawler

A quick-start helper to website crawling.

v1.1.0 2021-01-22 22:17 UTC

Requires

• php: >=5.4
• ext-curl: *
• ext-dom: *
• ext-json: *
• iggi/curlrequest: ^2

MIT

• Ignatios Drakoulas

• crawl
• data-mining

README

Installation

Install the latest version with

$ composer require iggi/crawler

Basic Usage

<?php
require_once "vendor/autoload.php";

use Iggi\Crawler;

class MyCrawler extends Crawler {

    protected $uri = "https://ignatisd.gr";

    public function __construct($proxy = null, $debug = 0)
    {
        parent::__construct($proxy, $debug);
    }

    public function hello() {
        $response = $this->curlRequest->get($this->getUrl("/hello"))->exec();
        if ($response->code === 200) {
            return json_decode($response->body, true);
        }
        return $this->errorHandler("Request failed");
    }
}


$crawler = new MyCrawler();
$result = $crawler->hello();

print_r($result);
/*
array(
    "success" => true,
    "message" => "Hello world!"
)
*/

Author

Ignatios Drakoulas - [e-mail] - https://twitter.com/ignatisd

License

Crawler is licensed under the MIT License - see the LICENSE file for details