a5730b701c
According to Wikipedia this is indeed right.. https://en.wikipedia.org/wiki/Robots_exclusion_standard#Allow_directive Then there are only many wrong examples out there.
5 lines
114 B
Plaintext
5 lines
114 B
Plaintext
# Might as well keep robots away for performance and privacy reasons.
|
|
User-agent: *
|
|
Allow: /index.php
|
|
Disallow: /
|