Noah Levitt
2398031010
let the OS pick an available port, to avoid what appear to be timing issues causing multiple browsers to choose the same port
2017-02-22 12:44:19 -08:00
Noah Levitt
23601e2e0a
Merge branch 'master' into qa
...
* master:
handle errors from extract-outlinks.js, which happens on polyvore.com because it changes the definition of Set 😭
2017-02-22 10:57:27 -08:00
Noah Levitt
3c4ab834da
handle errors from extract-outlinks.js, which happens on polyvore.com because it changes the definition of Set 😭
2017-02-22 10:57:11 -08:00
Noah Levitt
675e2f53b8
Merge branch 'master' into qa
...
* master:
brozzler-list-jobs --yaml
2017-02-16 10:21:01 -08:00
Noah Levitt
0d0da22613
brozzler-list-jobs --yaml
2017-02-16 10:20:36 -08:00
Noah Levitt
a0298898be
Merge branch 'master' into qa
...
* master:
missed this in the last commit
2017-02-15 23:21:06 -08:00
Noah Levitt
f02d4ed40e
missed this in the last commit
2017-02-15 23:20:47 -08:00
Noah Levitt
0ee14a8e6f
Merge branch 'master' into qa
...
* master:
deprecate current scope rule syntax and create new syntax with slightly different semantics (to be documented), and add parent_url_regex scope rule; unit test for scoping
2017-02-15 16:47:01 -08:00
Noah Levitt
b409e49cfa
deprecate current scope rule syntax and create new syntax with slightly different semantics (to be documented), and add parent_url_regex scope rule; unit test for scoping
2017-02-15 16:46:45 -08:00
Noah Levitt
f6fdb91d57
Merge branch 'master' into qa
...
* master:
add --yaml option to brozzler-list-* commands
take screenshot before running behavior (but after login) - thanks danielbicho
2017-02-15 23:13:32 +00:00
Noah Levitt
c0057e591a
add --yaml option to brozzler-list-* commands
2017-02-15 23:13:09 +00:00
Noah Levitt
1054e8e3cb
take screenshot before running behavior (but after login) - thanks danielbicho
2017-02-15 09:13:44 -08:00
Noah Levitt
08752a5163
Merge branch 'master' into qa
...
* master:
logging tweaks
2017-02-10 15:19:35 -08:00
Noah Levitt
e58f4b7c44
logging tweaks
2017-02-10 15:19:28 -08:00
Barbara Miller
f990adfa63
post-merge tidying
2017-02-05 22:03:28 -08:00
Barbara Miller
46ee409de8
Merge branch 'simpledo' into qa
2017-02-05 22:01:07 -08:00
Barbara Miller
d4fdd089a3
finish jinja2 update
2017-02-05 21:58:54 -08:00
Barbara Miller
91205e6108
isVisible
2017-02-05 21:58:54 -08:00
Noah Levitt
01f5343b48
fix behaviors.yaml, switch simpledo behavior to jinja2 template
2017-02-05 21:58:54 -08:00
Barbara Miller
3851daa27d
update for new behaviors set up
2017-02-05 21:58:54 -08:00
Barbara Miller
7296acdc19
simpledo first draft
2017-02-05 21:58:13 -08:00
Barbara Miller
1501d572da
isVisible
2017-02-05 21:56:25 -08:00
Noah Levitt
98144a2180
fix behaviors.yaml, switch simpledo behavior to jinja2 template
2017-02-05 21:56:25 -08:00
Barbara Miller
dcc2dae8c1
update for new behaviors set up
2017-02-05 21:56:24 -08:00
Barbara Miller
c627d7ee84
simpledo first draft
2017-02-05 21:56:24 -08:00
Barbara Miller
0ff6257bef
Merge branch 'qa' of github.com:internetarchive/brozzler into qa
2017-02-05 21:49:40 -08:00
Noah Levitt
aa22594928
Merge branch 'master' into qa
...
* master:
fix TypeError: not all arguments converted during string formatting
2017-02-03 17:24:53 -08:00
Noah Levitt
09fa41f959
fix TypeError: not all arguments converted during string formatting
2017-02-03 17:24:47 -08:00
Noah Levitt
7402657ae1
Merge branch 'master' into qa
...
* master:
make sure site is not "claimed" when it's finished
2017-02-03 16:40:22 -08:00
Noah Levitt
14e312e4c4
make sure site is not "claimed" when it's finished
2017-02-03 16:40:15 -08:00
Noah Levitt
3783e49cb4
Merge branch 'master' into qa
...
* master:
support for resuming jobs, keeping track of each start and stop time, used to enforce time limits correctly
let rethinkdb generate job.id if not supplied in configuration
2017-02-03 14:56:21 -08:00
Noah Levitt
a60878c5a7
support for resuming jobs, keeping track of each start and stop time, used to enforce time limits correctly
2017-02-03 14:56:12 -08:00
Noah Levitt
5a0301ac12
let rethinkdb generate job.id if not supplied in configuration
2017-02-03 14:53:50 -08:00
Noah Levitt
7f885758ae
Merge branch 'master' into qa
...
* master:
use underscore convention
2017-02-02 11:52:28 -08:00
Noah Levitt
129a1e8f47
use underscore convention
2017-02-02 11:52:19 -08:00
Noah Levitt
a19700b975
Merge branch 'master' into qa
...
* master:
improve TRACE level logging
2017-02-02 11:43:54 -08:00
Noah Levitt
5f4c5190da
improve TRACE level logging
2017-02-02 11:41:40 -08:00
Noah Levitt
87de7750e6
Merge branch 'master' into qa
...
* master:
stopgap fix for problem where an attempt to save a screenshot of a url with a hash tag containing spaces or non-ascii characters would fail, causing the whole brozzle of the page to fail, and end up in a retry loop (better handling of hash tags is planned which will obviate this change)
2017-02-01 22:39:29 +00:00
Noah Levitt
ed2d58d87d
stopgap fix for problem where an attempt to save a screenshot of a url with a hash tag containing spaces or non-ascii characters would fail, causing the whole brozzle of the page to fail, and end up in a retry loop (better handling of hash tags is planned which will obviate this change)
2017-02-01 22:39:12 +00:00
Noah Levitt
1874434d60
Merge branch 'master' into qa
...
* master:
pywb support for thumbnail: and screenshot: urls
fix oops
2017-01-31 10:26:48 -08:00
Noah Levitt
5c684779e5
pywb support for thumbnail: and screenshot: urls
2017-01-31 10:26:38 -08:00
Noah Levitt
8f5003b784
fix oops
2017-01-30 23:47:39 -08:00
Barbara Miller
6bf8cfe893
Merge branch 'qa' of github.com:internetarchive/brozzler into qa
2017-01-30 20:52:58 -08:00
Noah Levitt
8df96ceb59
Merge branch 'master' into qa
...
* master:
new flag Page.blocked_by_robots
be more patient to avoid spurious warnings waiting for browser to start up
2017-01-30 11:39:37 -08:00
Noah Levitt
4b6831b464
new flag Page.blocked_by_robots
2017-01-30 10:43:25 -08:00
Noah Levitt
a8b564f100
be more patient to avoid spurious warnings waiting for browser to start up
2017-01-24 10:06:37 -08:00
Noah Levitt
8c116295ea
Merge branch 'master' into qa
...
* master:
restore ping_timeout argument to WebSocketApp.run_forever to fix problem of leaking websocket receiver threads hanging forever on select()
missed a spot
improve brozzler-dashboard logging; fix default wayback baseurl in brozzler dashboard (https://github.com/internetarchive/brozzler/issues/31 ); tweak arg parsing related stuff
avoid js errors in case site or job is not configured to keep stats
add travis-ci slack notification to internetarchive/brozzler channel
2017-01-24 09:56:14 -08:00
Noah Levitt
d22cc075e0
restore ping_timeout argument to WebSocketApp.run_forever to fix problem of leaking websocket receiver threads hanging forever on select()
2017-01-24 09:55:56 -08:00
Noah Levitt
5375b819dd
missed a spot
2017-01-20 23:59:31 -08:00
Noah Levitt
c3b637d244
improve brozzler-dashboard logging; fix default wayback baseurl in brozzler dashboard ( https://github.com/internetarchive/brozzler/issues/31 ); tweak arg parsing related stuff
2017-01-20 23:41:59 -08:00
