Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-22 00:19:14 -04:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
brozzler/tests/htdocs/robots.txt
Noah Levitt 24cc8377fb robots.txt for testing
2016-11-16 12:12:17 -08:00

3 lines
31 B
Plaintext
Raw Permalink Blame History

User-agent: badbot
Disallow: /
Reference in New Issue View Git Blame Copy Permalink