Noah Levitt
5fdb2dd39c
documentation tweak
2019-05-16 14:03:43 -07:00
Noah Levitt
ee8ef23f0c
fix mistake in job-conf.rst
2019-04-30 10:49:48 -07:00
Karl-Rainer Blumenthal
bd78e07232
Copy edits to job-conf readme
...
Good reading and rampant pedantry!
2018-07-06 15:24:12 -04:00
Noah Levitt
a00b5a7fd5
explain brozzler use of warcprox_meta
2018-05-30 18:06:39 -07:00
Noah Levitt
de1f240e25
describe scope rule conditions
...
plus a bunch of tweaks and fixes
2018-05-15 11:01:09 -07:00
Noah Levitt
a327cb626f
more explication of scoping
2018-05-14 17:31:45 -07:00
Noah Levitt
2cf474aa1d
update docs to match new seed ssurt behavior
2018-05-14 16:59:55 -07:00
Noah Levitt
85a4757527
s/max_hops_off_surt/max_hops_off/
2018-05-14 15:38:28 -07:00
Noah Levitt
98ce67ef36
WIP some words on scoping
2018-05-14 15:38:28 -07:00
Noah Levitt
88214236bb
WIP starting to flesh out "scoping" section
2018-05-14 15:38:28 -07:00
Noah Levitt
6df2c1cf22
WIP some explanation of automatic login
2018-05-14 15:38:28 -07:00
Noah Levitt
914289b414
WIP documentation!
2018-05-14 15:38:28 -07:00
Noah Levitt
f26d711a89
new job setting max_claimed_sites
...
Puts a cap on the number of sites belonging to a given job that can be brozzled
simultaneously across the cluster. Addresses the problem of a job with many
seeds starving out other jobs. For AITFIVE-1578.
2018-03-01 17:17:54 -08:00
Noah Levitt
29b19b1e9d
shed some light on the travis-ci error
2017-06-23 13:56:25 -07:00
Noah Levitt
934190084c
Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see https://github.com/internetarchive/warcprox/commit/8caae0d7d3 ), and enables warcprox features if so.
2017-03-24 13:55:23 -07:00
Noah Levitt
5a0301ac12
let rethinkdb generate job.id if not supplied in configuration
2017-02-03 14:53:50 -08:00
Alex Osborne
743b5a4347
Add user_agent option
...
Currently doesn't apply to requests made by youtube-dl as I
couldn't see a thread-safe way of doing that.
2016-10-05 04:25:09 +11:00
Noah Levitt
bfd4c1f8c6
document a bunch of job settings
2016-09-29 16:15:44 -07:00
Noah Levitt
8c9a9c5666
starting on documenting job configuration
2016-09-29 12:03:16 -07:00
