scrapekit/scrapekit

Web Scraping Framework

0.1.0 2020-02-23 21:39 UTC

Requires

• afterflow/recipe: 0.1.*
• guzzlehttp/guzzle: ^6.5
• illuminate/support: 5.*|6.*|7.*
• imangazaliev/didom: 1.14.*
• ramsey/uuid: 3.9.*
• symfony/css-selector: 5.*|4.*
• symfony/dom-crawler: 5.*|4.*
• textalk/websocket: ^1.2

Requires (Dev)

• phpunit/phpunit: 8.*
• squizlabs/php_codesniffer: ^3.5
• wanfeiyy/dd: ^1.0

MIT

• librevlad

README

Top secret project.

Installation

composer require scrapekit/scrapekit

Browser Automation

ScrapeKit Browser uses Chrome DevTools Protocol.

    $tab = scrapekit()->chrome('http://localhost:9222')->tabs()->new();

    $tab->geo( 45, 21 )
        ->navigate( 'https://mylocation.org/' )
        ->find( '[aria-controls="geo-div"]' )->click()
        ->tab()
        ->find( '#geo-test' )->click()
        ->tab()
        ->pause( 100 )
        ->geo()
        ->navigate( 'https://mylocation.org/' )
        ->find( '[aria-controls="geo-div"]' )->click()
        ->tab()
        ->find( '#geo-test' )->click();

    $tab->pause( 100 );

    $lat = $tab->find( '#geo-latitude' )->text();
    $lng = $tab->find( '#geo-longitude' )->text();

    $tab->close();

HTTP Automation

ScrapeKit HTTP uses Guzzle, exposing a simple and powerful API with on-demand complexity.

Basic Usage

Make a single GET request and grab the response:

$html = scrapekit()->http()->request('http://httpbin.org')->body();
$headers = scrapekit()->http()->request('http://httpbin.org/anything')->headers()->all();

Use Request object and parse JSON response:

$array = scrapekit()->http()->request( Request::make('http://httpbin.org/anything')->parseResponseWith( JsonParser::class ) )->parse()->data();

Batch Requests and Async

TODO

DomQL

TODO

Contributing

TODO