Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-02-23 16:19:49 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
brozzler/tests/htdocs/robots.txt
Noah Levitt 24cc8377fb robots.txt for testing
2016-11-16 12:12:17 -08:00

3 lines
31 B
Plaintext
Raw Blame History

User-agent: badbot
Disallow: /
Reference in New Issue View Git Blame Copy Permalink