Noah Levitt 405c5725e4 restore reclamation of orphaned, claimed sites, and heartbeat site.last_claimed every 7 minutes during youtube-dl processing, to prevent another brozzler-worker claiming the site

2017-06-23 13:50:49 -07:00

..

dashboard

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

js-templates

add relocated behavior file with updated copyright

2017-05-22 12:38:50 -07:00

__init__.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

behaviors.yaml

simpleclicks for recent issuu.com URLs

2017-06-09 14:08:34 -07:00

browser.py

updates per Noah's review

2017-06-02 13:27:01 -07:00

chrome.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

cli.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

easy.py

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

frontier.py

restore reclamation of orphaned, claimed sites, and heartbeat site.last_claimed every 7 minutes during youtube-dl processing, to prevent another brozzler-worker claiming the site

2017-06-23 13:50:49 -07:00

job_schema.yaml

Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see https://github.com/internetarchive/warcprox/commit/8caae0d7d3), and enables warcprox features if so.

2017-03-24 13:55:23 -07:00

model.py

consolidate job.py and site.py into model.py, and let Job and Site share the elapsed() method by way of a mixin

2017-03-29 18:49:04 -07:00

pywb.py

use urlcanon library for canonicalization, surtification, scope match rules

2017-03-15 14:59:51 -07:00

robots.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

worker.py

restore reclamation of orphaned, claimed sites, and heartbeat site.last_claimed every 7 minutes during youtube-dl processing, to prevent another brozzler-worker claiming the site

2017-06-23 13:50:49 -07:00