Demander aux robots de ne pas crawler les archives
This commit is contained in:
parent
a59696a60e
commit
7ac19b0076
4
robots.txt
Normal file
4
robots.txt
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
User-agent: *
|
||||||
|
Disallow: /*/*/archive/*.tar.gz
|
||||||
|
Disallow: /*/*/archive/*.zip
|
||||||
|
Disallow: /*/*/archive/*.bundle
|
Loading…
Reference in New Issue
Block a user