paste.chapril.org-privatebin/robots.txt

# Might as well keep robots away for performance and privacy reasons.
User-agent: *
Allow: /index.php
Disallow: /
For server-load and privacy reasons, disallow robots. 2012-08-02 17:23:04 +02:00			`# Might as well keep robots away for performance and privacy reasons.`
			`User-agent: *`
Change order of allow rule in robots.txt I think this is the proper order. Don't know if it matters, but it is not bad to change it in any case. 2016-07-03 13:05:26 +02:00			`Allow: /index.php`
Revert order change According to Wikipedia this is indeed right.. https://en.wikipedia.org/wiki/Robots_exclusion_standard#Allow_directive Then there are only many wrong examples out there. 2016-07-03 13:09:47 +02:00			`Disallow: /`