brozzler/requirements.txt

kombu
websocket-client
argparse
PyYAML
git+https://github.com/ikreymer/surt.git@py3
youtube_dl
git+https://github.com/seomoz/reppy.git
basic of site/seed crawling with scoping 2015-07-09 18:36:07 -07:00			`kombu`
support adding extra http request headers 2015-07-17 13:45:27 -07:00			`websocket-client`
basic of site/seed crawling with scoping 2015-07-09 18:36:07 -07:00			`argparse`
			`PyYAML`
			`git+https://github.com/ikreymer/surt.git@py3`
determine if youtube-dl can do something with a url 2015-07-13 16:40:56 -07:00			`youtube_dl`
honor site proxy setting; remove brozzler-worker options that are now configured at the site level (and in the case of ignore_cert_errors, always on, no longer an option); use "reppy" library for robots.txt handling; fix some bugs 2015-07-16 17:19:12 -07:00			`git+https://github.com/seomoz/reppy.git`