PHP
Web crawling and scraping library.
Latest version requires crwlr/url ^2.1
11 477 367
Robots Exclusion Standard/Protocol Parser for Web Crawling/Scraping
Latest version requires crwlr/url ^1.0|^2.0
19 042 10