Packages from crwlr

  • PHP

    crwlr/crawler

    Web crawling and scraping library.

    2 701 298

  • PHP

    crwlr/crawler-ext-browser

    Extension for the crwlr/crawler package containing steps utilizing a headless browser.

    131 0

  • PHP

    crwlr/crwl-ext-browser

    Extension configurations for integration of crwlr/crawler-ext-browser into the crwl.io app.

    92 0

  • PHP

    crwlr/crwl-extension-utils

    Utils for extension packages for the crwl.io app.

    325 0

  • PHP

    crwlr/html-2-text

    Convert HTML to formatted plain text.

    1 046 2

  • PHP

    crwlr/query-string

    A library for convenient handling of query strings used in HTTP requests.

    1 002 626 15

  • PHP

    crwlr/robots-txt

    Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping

    5 175 8

  • PHP

    crwlr/schema-org

    Extract schema.org structured data from HTML documents.

    3 461 10

  • PHP

    crwlr/url

    Swiss Army knife for URLs.

    29 708 95

  • PHP

    crwlr/utils

    Utilities that are needed in multiple crawler packages.

    2 651 2