Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-05-02 06:36:20 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
brozzler/tests/htdocs/robots.txt
Noah Levitt 24cc8377fb robots.txt for testing
2016-11-16 12:12:17 -08:00

2 lines
31 B
Text
Raw Blame History

User-agent: badbot
Disallow: /
Reference in a new issue View git blame Copy permalink