LogoopenSUSE Build Service > Projects
Sign Up | Log In

database of robots.txt-derived permissions

This module parses _/robots.txt_ files as specified in "A Standard for
Robot Exclusion", at <http://www.robotstxt.org/wc/norobots.html> Webmasters
can use the _/robots.txt_ file to forbid conforming robots from accessing
parts of their web site.

Source Files (show merged sources derived from linked package)

Filename Size Changed Actions
WWW-RobotRules-6.02.tar.gz 8.85 KB Download File
perl-WWW-RobotRules.changes 344 Bytes Download File
perl-WWW-RobotRules.spec 2.85 KB Download File

Comments for home:jayvdb:perl-critic (0)

Login required, please login or signup in order to comment