Vangelis Banos 820c7cd8cc Fix to delete tmpdir on Chrome.stop() ...

The ``self._home_tmpdir.cleanup()`` cmd is not always executed when
stopping Chrome. As a result, a large number of ``/tmp/tmpXXX`` dirs are
created in production.

The reason is that ``Chrome.stop()`` execution can stop in the ``return``
statement in the following line:
https://github.com/internetarchive/brozzler/blob/master/brozzler/chrome.py#L268
and ``cleanup()`` does not run.

Moving the ``cleanup()`` in the ``finally`` part of the
``try/catch/finally`` block makes it run always in the end of
``Chrome.stop()`` and cleans up the tmp directory in any case.

2018-01-15 13:09:43 +00:00

..

dashboard

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

js-templates

Update SoundCloud.com behavior selectors.

2017-08-18 14:16:51 -07:00

__init__.py

Replace cwd var with d

2018-01-09 06:33:03 +00:00

behaviors.yaml

simpleclicks for ARI-5241

2017-11-20 17:25:32 -08:00

browser.py

Configurable JS templates location

2018-01-04 17:37:02 +00:00

chrome.py

Fix to delete tmpdir on Chrome.stop()

2018-01-15 13:09:43 +00:00

cli.py

fix brozzler-list-sites --site=SITE_ID

2017-12-21 17:16:41 -08:00

easy.py

parameterize command line entry points and add tests of --version, a rudimentary check that the commands at least run

2017-04-14 11:46:26 -07:00

frontier.py

quiet down the logging

2017-12-15 15:57:36 -08:00

job_schema.yaml

Refactor the way the proxy is configured. Job/site settings "proxy" and "enable_warcprox_features" are gone. Brozzler-worker now has mutually exclusive options --proxy and --warcprox-auto. --warcprox-auto means find an instance of warcprox in the service registry, and enable warcprox features. --proxy is provided, determines if proxy is warcprox by consulting http://{proxy_address}/status (see https://github.com/internetarchive/warcprox/commit/8caae0d7d3), and enables warcprox features if so.

2017-03-24 13:55:23 -07:00

model.py

enforce time limits based on time claimed by worker actively brozzling, to avoid problem of stopping crawls that haven't had much chance to crawl, because of cluster busy-ness

2017-06-26 18:00:32 -07:00

pywb.py

use urlcanon library for canonicalization, surtification, scope match rules

2017-03-15 14:59:51 -07:00

robots.py

use %r instead of calling repr()

2017-06-07 13:07:42 -07:00

worker.py

set a timeout on warcprox_write_record request

2017-12-27 15:52:55 -08:00