LogoopenSUSE Build Service > Projects
Sign Up | Log In

database of robots.txt-derived permissions

This module parses _/robots.txt_ files as specified in "A Standard for
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.


Source Files (show merged sources derived from linked package)

Filename Size Changed Actions
WWW-RobotRules-6.02.tar.gz 8.85 KB over 6 years ago Download File
perl-WWW-RobotRules.changes 344 Bytes over 6 years ago Download File
perl-WWW-RobotRules.spec 2.85 KB over 6 years ago Download File

Comments for home:jayvdb:perl-critic (0)