opportus/web-scraper

A web scraper taking as arguments a list of URIs and a list of XPath queries to perform on each document. Returns an instance of DataInterface.

v2.0.0 2018-12-29 11:08 UTC

Requires

• php: ^7.2
• fabpot/goutte: ^4.1

MIT

• opportus

README

A web scraper taking as arguments a list of URIs and a list of XPath queries to perform on each document. Returns an instance of DataInterface.

Installation

$ composer require opportus/web-scraper

Usage

Use Opportus\WebScraper\WebScraper;

$uris = [
    'https://en.wikipedia.org/wiki/Web_scraping',
    'https://en.wikipedia.org/wiki/XPath',
];

$queries = [
    '//p[1][node()]',
    '//div[@id="mw-normal-catlinks"]/ul//li[node()]',
];

$scraper = new WebScraper();

$data = $scraper->scrap($uris, $queries); // @see https://github.com/opportus/web-scraper/blob/master/src/DataInterface.php