Other approach was too slow and caused db contention.
New approach avoids (slow) rethinkdb join by max_claimed_sites job
parameter to each of the job's sites. Uses rethinkdb fold() to count
claimed sites and enforce max_claimed_sites within a single query.
By default chromium creates multiple renderer processes (each running
multiple threads) for each instance of a site the user visits. What we
see from `ps auxcf` output is the following:
```
\_ chromium-browse
\_ chromium-browse
| \_ chromium-browse
| \_ chromium-browse
| \_ chromium-browse
| \_ chromium-browse
```
Using the `--single-process` option, we run all renderers in the same
process, saving the overhead of running multiple processes. `ps auxcf`
output is the following:
```
\_ chromium-browse
\_ chromium-browse
\_ chromium-browse
```
Performance is improved a bit and I guess that using this in large scale
Brozzler deployments will have even better performance effects.
The potential problem of `--single-process` is stability (if a renderer
crashes, the whole browser also crashes) but since we use very short-lived
instances of chromium, we don't worry about this.
Details on chromium process models:
https://www.chromium.org/developers/design-documents/process-models
Puts a cap on the number of sites belonging to a given job that can be brozzled
simultaneously across the cluster. Addresses the problem of a job with many
seeds starving out other jobs. For AITFIVE-1578.
Every time we run a JS behavior, we load a Jinja2 template.
By default, Jinja2 has option `auto_reload=True`. This mean that
every time a template is requested the loader checks if the source file changed
and if yes, it will reload the template. For higher performance it’s possible
to disable that.
Also note that Jinja caches 400 templates by default.
Ref: http://jinja.pocoo.org/docs/2.10/api/
In Brozzler, we don't make changes to JS templates while the system is
running. So, there is no point in having auto_reload=True.
