Noah Levitt 64211475c0 lengthen site session brozzling time to 15 minutes ...

This should reduce contention over the "sites" table, which should help
keep more available browsers busy across the cluster.

2018-01-29 15:34:54 -08:00

..

dashboard

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

js-templates

update copyright dates

2018-01-15 19:39:41 -08:00

__init__.py

Replace cwd var with d

2018-01-09 06:33:03 +00:00

behaviors.yaml

update copyright dates

2018-01-15 19:39:41 -08:00

browser.py

Merge pull request #75 from galgeek/pageInterstitialShown

2018-01-26 16:18:22 -08:00

chrome.py

Fix to delete tmpdir on Chrome.stop()

2018-01-15 13:09:43 +00:00

cli.py

--warcprox-auto distribute assigned sites evenly (#78)

2018-01-19 14:54:33 -08:00

easy.py

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

frontier.py

quiet down the logging

2017-12-15 15:57:36 -08:00

job_schema.yaml

Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see https://github.com/internetarchive/warcprox/commit/8caae0d7d3), and enables warcprox features if so.

2017-03-24 13:55:23 -07:00

model.py

enforce time limits based on time claimed by worker actively brozzling, to avoid problem of stopping crawls that haven't had much chance to crawl, because of cluster busy-ness

2017-06-26 18:00:32 -07:00

pywb.py

use urlcanon library for canonicalization, surtification, scope match rules

2017-03-15 14:59:51 -07:00

robots.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

worker.py

lengthen site session brozzling time to 15 minutes

2018-01-29 15:34:54 -08:00