1382 Commits

Author SHA1 Message Date
Barbara Miller
6290692ac4 Merge branch 'master' of github.com:internetarchive/brozzler 2020-12-02 14:34:30 -08:00
Barbara Miller
0f27c9995a
bump version 2020-10-29 17:12:14 -07:00
jkafader
5005c619f6
Merge pull request #211 from internetarchive/galgeek-websocket-url-timeout
configurable websocket url timeout, default 60
2020-10-29 17:08:48 -07:00
Barbara Miller
11c5cfa865 add param for Chrome.start 2020-10-21 15:39:46 -07:00
Barbara Miller
2d380cc8f2 Merge branch 'master' of github.com:internetarchive/brozzler 2020-10-14 22:06:11 -07:00
Barbara Miller
dc50fe1db2
Merge pull request #212 from internetarchive/bump-version-to-1.5.23
bump version after merge
2020-10-13 15:21:18 -07:00
Barbara Miller
052c3552ca
bump version after merge 2020-10-13 15:19:50 -07:00
Barbara Miller
f2ebdca597
configurable websocket url timeout, default 60 2020-10-13 15:12:32 -07:00
Barbara Miller
bb7594a14d
Merge pull request #209 from vbanos/outlinks-timeout
Thanks, @vbanos!
2020-10-13 15:01:55 -07:00
Barbara Miller
4372752bd6 Merge branch 'qa' of github.com:internetarchive/brozzler 2020-10-05 14:37:07 -07:00
Vangelis Banos
8addaf31d5 Add option extract_outlinks_timeout
`Browser.extract_outlinks` has a default `timeout=60` parm that cannot be
changed in any way. (It is always invoked using `extract_outlinks()`.

We add param `extract_outlinks_timeout=60` to `BrozzlerWorker` and
`Browser.browse_page` to allow that.
2020-10-04 15:39:30 +00:00
Barbara Miller
18d3f5f930
Merge pull request #208 from internetarchive/galgeek-patch-2
based on PR #207 — thanks @cclaus!
2020-09-21 18:06:03 -07:00
Barbara Miller
297eaac6dd
update travis.yml and test! 2020-09-21 17:08:39 -07:00
Barbara Miller
c744bb2f92
update copyright 2020-09-01 19:05:21 -07:00
Barbara Miller
d599778c27
Merge pull request #206 from internetarchive/galgeek-patch-1
bump version after merge
2020-08-05 09:24:28 -07:00
Barbara Miller
84d6bb43fa
bump version after merge 2020-08-05 09:23:58 -07:00
Barbara Miller
5a6ecb09d5
Merge pull request #205 from vbanos/behavior-timeout-zero
Skip loading behavior when behavior_timeout=0

behavior_timeout is an existing parameter to `Browser.browse_page`
2020-08-04 16:18:58 -07:00
Neil Minton
12913cccf0
Merge pull request #204 from galgeek/noplaylist-ydl
youtube-dl option noplaylist: True
2020-08-04 14:12:14 -04:00
Vangelis Banos
8b10587031 Skip loading behavior when behavior_timeout=0
The user may set `behavior_timeout=0`. This means that they don't want
to run the behavior. As it is now, Brozzler will invoke
`brozzler.behavior_script` to load the script and `self.run_behavior`
to execute it.
We will run the behavior using `Runtime.evaluate` but then it will be
terminated immediately because of timeout=0.

It is better to skip behavior loading and running when
`behavior_timeout=0`.
2020-08-04 06:27:21 +00:00
Barbara Miller
dc0d99470a
Merge pull request #203 from miku/update-readme-proxy
Thank you, @miku!
2020-07-28 13:43:19 -07:00
Martin Czygan
8e670ca814 readme: remove proxy from job configuration
It has been removed in 934190084c73699747cf3f4c4d2ee7e268927eae.
2020-07-28 22:21:05 +02:00
Barbara Miller
fa2fe1eadf Merge branch 'noplaylist-ydl' into qa 2020-07-24 16:24:36 -07:00
Barbara Miller
e3a067cf60 youtube-dl option noplaylist: True 2020-07-24 16:22:50 -07:00
jkafader
1b9ebca13c
Merge pull request #202 from galgeek/limit_downloadThroughput
configurable limit for Chromium download throughput
2020-07-23 14:14:20 -07:00
Barbara Miller
c482dc4d3b Merge branch 'fix_test_ydl_stitching' into qa 2020-07-14 10:53:40 -07:00
Barbara Miller
7f2d3485a6 try waiting a little 2020-07-14 10:53:07 -07:00
Barbara Miller
35b239c984 Merge branch 'limit_downloadThroughput' into qa 2020-07-14 10:13:17 -07:00
Barbara Miller
739d09294e make configurable 2020-07-14 10:12:28 -07:00
Barbara Miller
36b4f80350 try SPN2 downloadThroughput limit 2020-07-14 10:12:28 -07:00
Barbara Miller
b6bf392773 Merge branch 'limit_downloadThroughput' into qa 2020-07-11 14:17:16 -07:00
Barbara Miller
d40bcb0497 Revert "travis_wait"
This reverts commit f462f9c8cdff1bad7fdded14169b80393dfeb4a2.
2020-07-11 14:14:26 -07:00
Barbara Miller
5329218f22
Un-Update .travis.yml 2020-07-09 14:39:33 -07:00
Barbara Miller
17ea3ea1c5
Update .travis.yml 2020-07-09 12:46:04 -07:00
Barbara Miller
c86ad357a8 Merge branch 'limit_downloadThroughput' into qa 2020-07-09 12:37:53 -07:00
Barbara Miller
f462f9c8cd travis_wait 2020-07-09 12:36:55 -07:00
Barbara Miller
98d624acbe Merge branch 'limit_downloadThroughput' into qa 2020-07-09 12:28:37 -07:00
Barbara Miller
dc1b809a6c travis_wait 2020-07-09 12:26:15 -07:00
Barbara Miller
bef2730e7e Merge branch 'limit_downloadThroughput' into qa 2020-06-27 10:22:48 -07:00
Barbara Miller
4936ce09d7 make configurable 2020-06-26 18:32:11 -07:00
Barbara Miller
754d024f7a update (C) 2020-06-26 11:49:27 -07:00
Barbara Miller
96696099f6 Merge branch 'limit_downloadThroughput' into qa 2020-06-25 17:26:11 -07:00
Barbara Miller
6c6765e457 MB, not GB 2020-06-25 17:25:56 -07:00
Barbara Miller
6f9fc87323 Merge branch 'limit_downloadThroughput' into qa 2020-06-25 17:19:39 -07:00
Barbara Miller
8481dfac9c try SPN2 downloadThroughput limit 2020-06-25 17:18:56 -07:00
Barbara Miller
2ee7f71db5 Merge branch 'error_ydl_playlist' into qa 2020-06-23 13:32:41 -07:00
Barbara Miller
9138e9c91d typo: %s not {} 2020-06-23 13:32:18 -07:00
Barbara Miller
76d3a4fc5f not the problem 2020-06-23 13:18:16 -07:00
Barbara Miller
5f37c72db4 Merge branch 'error_ydl_playlist' into qa 2020-06-23 13:14:28 -07:00
Barbara Miller
99d03e2544 try / except 2020-06-23 13:13:58 -07:00
Barbara Miller
03594413f9
Merge pull request #200 from NGTmeaty/fix-test
Merging for the current fixes—thanks, @NGTmeaty!
2020-06-18 13:22:41 -07:00