Noah Levitt e5371ef0b0 Merge branch 'master' into qa ...

* master:
  recognize ConnectionError (of which ConnectionResetError is a subclass) in _warcprox_write_record as a proxy error

2017-05-12 10:04:06 -07:00

..

dashboard

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

js-templates

Merge branch 'master' into qa

2017-04-05 12:09:58 -07:00

__init__.py

refactor thread_raise safety to use a context manager

2017-04-24 19:51:51 -07:00

behaviors.yaml

Merge branch 'ARI-4868' into qa

2017-05-10 15:09:00 -07:00

browser.py

Merge branch 'master' into qa

2017-05-01 13:00:34 -07:00

chrome.py

let the OS pick an available port, to avoid what appear to be timing issues causing multiple browsers to choose the same port

2017-02-22 12:44:19 -08:00

cli.py

do a better job of making sure to shut down the browser when brozzle-page is killed

2017-05-03 16:43:31 -07:00

easy.py

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

frontier.py

re-claim sites after 1 hour instead of 2 so that sites don't have to wait as long to be brozzled again in case of kill -9 brozzler-worker

2017-05-01 13:00:04 -07:00

job_schema.yaml

Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see https://github.com/internetarchive/warcprox/commit/8caae0d7d3), and enables warcprox features if so.

2017-03-24 13:55:23 -07:00

model.py

consolidate job.py and site.py into model.py, and let Job and Site share the elapsed() method by way of a mixin

2017-03-29 18:49:04 -07:00

pywb.py

use urlcanon library for canonicalization, surtification, scope match rules

2017-03-15 14:59:51 -07:00

robots.py

raise brozzler.ProxyError in case of proxy error fetching robots.txt, doing youtube-dl, or doing raw fetch

2017-04-17 18:15:22 -07:00

worker.py

recognize ConnectionError (of which ConnectionResetError is a subclass) in _warcprox_write_record as a proxy error

2017-05-12 10:03:53 -07:00