webignition/robots-txt-parser

parserrobots.txt

Overall: 4 508 installs
30 days: 997 installs
Today: 0 installs

Parser for robots.txt files

Maintainer: webignition
Homepage: https://github.com/webignition/robots-txt-parser
Canonical: https://github.com/webignition/robots-txt-parser
Source: https://github.com/webignition/robots-txt-parser/tree/master
Issues: https://github.com/webignition/robots-txt-parser/issues