Noah Levitt 5be7dd4407 Merge branch 'master' into qa ...

* master:
  bump dev version number after some PR merges
  bugfix for BrozzlerWorker._needs_browsing
  Remove redundant method parameter.
  bugfix
  Make youtube-dl optional in BrozzlerWorker.brozzle_page

2017-08-01 12:05:07 -07:00

..

dashboard

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

js-templates

Merge pull request #55 from galgeek/ARI-5242

2017-08-01 12:04:18 -07:00

__init__.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

behaviors.yaml

Merge pull request #55 from galgeek/ARI-5242

2017-08-01 12:04:18 -07:00

browser.py

Merge branch 'master' into qa

2017-08-01 12:05:07 -07:00

chrome.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

cli.py

Hide the options using argparse.SUPPRESS

2017-07-06 06:25:04 +00:00

easy.py

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

frontier.py

enforce time limits based on time claimed by worker actively brozzling, to avoid problem of stopping crawls that haven't had much chance to crawl, because of cluster busy-ness

2017-06-26 18:00:32 -07:00

job_schema.yaml

Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see https://github.com/internetarchive/warcprox/commit/8caae0d7d3), and enables warcprox features if so.

2017-03-24 13:55:23 -07:00

model.py

enforce time limits based on time claimed by worker actively brozzling, to avoid problem of stopping crawls that haven't had much chance to crawl, because of cluster busy-ness

2017-06-26 18:00:32 -07:00

pywb.py

use urlcanon library for canonicalization, surtification, scope match rules

2017-03-15 14:59:51 -07:00

robots.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

worker.py

bugfix for BrozzlerWorker._needs_browsing

2017-08-01 11:43:42 +00:00