brozzler/brozzler
2017-08-18 14:16:51 -07:00
..
dashboard parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run 2017-04-14 11:46:26 -07:00
js-templates Generalize default behavior. 2017-08-18 14:16:51 -07:00
__init__.py use %r instead of calling repr() 2017-06-07 13:07:42 -07:00
behaviors.yaml Merge pull request #60 from galgeek/ARI-5453 2017-08-14 17:14:09 -07:00
browser.py Remove redundant method parameter. 2017-07-31 12:36:17 +00:00
chrome.py use %r instead of calling repr() 2017-06-07 13:07:42 -07:00
cli.py Hide the options using argparse.SUPPRESS 2017-07-06 06:25:04 +00:00
easy.py parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run 2017-04-14 11:46:26 -07:00
frontier.py enforce time limits based on time claimed by worker actively brozzling, to avoid problem of stopping crawls that haven't had much chance to crawl, because of cluster busy-ness 2017-06-26 18:00:32 -07:00
job_schema.yaml Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see 8caae0d7d3), and enables warcprox features if so. 2017-03-24 13:55:23 -07:00
model.py enforce time limits based on time claimed by worker actively brozzling, to avoid problem of stopping crawls that haven't had much chance to crawl, because of cluster busy-ness 2017-06-26 18:00:32 -07:00
pywb.py use urlcanon library for canonicalization, surtification, scope match rules 2017-03-15 14:59:51 -07:00
robots.py use %r instead of calling repr() 2017-06-07 13:07:42 -07:00
worker.py Merge pull request #54 from vbanos/worker_on_request 2017-08-03 14:17:02 -07:00