Search by 
Packages from tomverran · Transparency log
-
-
PHP
tomverran/robots-txt-checker
Given a robots.txt file, user agent and URL path will tell you whether you're allowed to access a page