Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-08-03 12:06:28 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

proxy is not supposed to have http:// prefix

Browse source 
Looks like the prefixes are added by BrozzleWorker._fetch_url()
This commit is contained in:
Alex Osborne 2016-05-17 16:20:38 +10:00
parent 02af30edd4
commit 484805fbda
1 changed files with 2 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

4
README.rst
View file

@ -46,14 +46,14 @@ Job Configuration
----------------- -----------------
Jobs are defined using yaml files. Options may be specified either at the Jobs are defined using yaml files. Options may be specified either at the
top-level or on individual seeds. A job id id and at least one seed url top-level or on individual seeds. A job id and at least one seed url
must be specified, everything else is optional. must be specified, everything else is optional.
:: ::
id: myjob id: myjob
time_limit: 60 # seconds time_limit: 60 # seconds
proxy: http://127.0.0.1:8000 # point at warcprox for archiving proxy: 127.0.0.1:8000 # point at warcprox for archiving
ignore_robots: false ignore_robots: false
enable_warcprox_features: false enable_warcprox_features: false
warcprox_meta: null warcprox_meta: null