922 Commits

Author SHA1 Message Date
Noah Levitt
2aacf01950 Merge branch 'master' into qa
* master:
  add the new urlcanon.MatchRule conditions to job_schema.yaml
2017-03-15 17:08:37 -07:00
Noah Levitt
0021a9d5f0 add the new urlcanon.MatchRule conditions to job_schema.yaml 2017-03-15 17:08:27 -07:00
Noah Levitt
63474c09f2 Merge branch 'master' into qa
* master:
  use urlcanon library for canonicalization, surtification, scope match rules
  more automated tests of frontier stuff
2017-03-15 15:00:01 -07:00
Noah Levitt
12fb9eaa15 use urlcanon library for canonicalization, surtification, scope match rules 2017-03-15 14:59:51 -07:00
Noah Levitt
479f0f7e09 more automated tests of frontier stuff 2017-03-15 14:54:16 -07:00
Noah Levitt
6526c40bb8 Merge branch 'master' into qa
* master:
  turns out we want populate_defaults to happen in __init__, fix so things work right
2017-03-08 17:34:34 -08:00
Noah Levitt
9e1e002a71 turns out we want populate_defaults to happen in __init__, fix so things work right 2017-03-07 17:52:38 -08:00
Noah Levitt
335b67f42b Merge branch 'master' into qa
* master:
  use updated doublethink library populate_defaults() to avoid problem where under certain circumstances field values from the database would be overwritten by defaults
2017-03-07 13:20:10 -08:00
Noah Levitt
01653c01d7 use updated doublethink library populate_defaults() to avoid problem where under certain circumstances field values from the database would be overwritten by defaults 2017-03-07 13:19:56 -08:00
Noah Levitt
59316a38f8 Merge branch 'master' into qa
* master:
  fix bug with seed redirects where scope change was applied too late to affect scoping of outlinks from the seed (with automated tests)
2017-03-06 15:13:48 -08:00
Noah Levitt
242ff51ec7 fix bug with seed redirects where scope change was applied too late to affect scoping of outlinks from the seed (with automated tests) 2017-03-06 15:13:40 -08:00
Noah Levitt
d55dd0b26f Merge branch 'master' into qa
* master:
  add tests of backwards compatibility handling of start/stop times and fix a bug or two
2017-03-02 16:53:31 -08:00
Noah Levitt
40bbbb3524 add tests of backwards compatibility handling of start/stop times and fix a bug or two 2017-03-02 16:53:24 -08:00
Noah Levitt
c434bbf3da Merge branch 'master' into qa
* master:
  rethinkstuff is now "doublethink
2017-03-02 12:48:58 -08:00
Noah Levitt
569af05b11 rethinkstuff is now "doublethink 2017-03-02 12:48:45 -08:00
Noah Levitt
95f362d49a Merge branch 'master' into qa
* master:
  use new rethinkstuff ORM
2017-02-28 16:12:58 -08:00
Noah Levitt
700b08b7d7 use new rethinkstuff ORM 2017-02-28 16:12:50 -08:00
Noah Levitt
6db69de2e5 Merge branch 'master' into qa
* master:
  fix issue where use of YoutubeDLSpy caused youtube-dl connections to remote servers to be kept open
2017-02-24 11:15:34 -08:00
Noah Levitt
a1f1681cad fix issue where use of YoutubeDLSpy caused youtube-dl connections to remote servers to be kept open 2017-02-24 11:15:17 -08:00
Noah Levitt
2b41cebfc1 Merge branch 'master' into qa
* master:
  fix typo
2017-02-23 10:47:10 -08:00
Noah Levitt
b4f19e2594 fix typo 2017-02-23 10:47:04 -08:00
Noah Levitt
b496bce320 Merge branch 'master' into qa
* master:
  more pywb monkey-patching to get at least some youtube videos captured by brozzler to play back
2017-02-23 10:43:15 -08:00
Noah Levitt
7417310d57 more pywb monkey-patching to get at least some youtube videos captured by brozzler to play back 2017-02-23 10:43:07 -08:00
Noah Levitt
cb75bb6e04 Merge branch 'master' into qa
* master:
  let the OS pick an available port, to avoid what appear to be timing issues causing multiple browsers to choose the same port
2017-02-22 12:44:27 -08:00
Noah Levitt
2398031010 let the OS pick an available port, to avoid what appear to be timing issues causing multiple browsers to choose the same port 2017-02-22 12:44:19 -08:00
Noah Levitt
23601e2e0a Merge branch 'master' into qa
* master:
  handle errors from extract-outlinks.js, which happens on polyvore.com because it changes the definition of Set 😭
2017-02-22 10:57:27 -08:00
Noah Levitt
3c4ab834da handle errors from extract-outlinks.js, which happens on polyvore.com because it changes the definition of Set 😭 2017-02-22 10:57:11 -08:00
Noah Levitt
675e2f53b8 Merge branch 'master' into qa
* master:
  brozzler-list-jobs --yaml
2017-02-16 10:21:01 -08:00
Noah Levitt
0d0da22613 brozzler-list-jobs --yaml 2017-02-16 10:20:36 -08:00
Noah Levitt
a0298898be Merge branch 'master' into qa
* master:
  missed this in the last commit
2017-02-15 23:21:06 -08:00
Noah Levitt
f02d4ed40e missed this in the last commit 2017-02-15 23:20:47 -08:00
Noah Levitt
0ee14a8e6f Merge branch 'master' into qa
* master:
  deprecate current scope rule syntax and create new syntax with slightly different semantics (to be documented), and add parent_url_regex scope rule; unit test for scoping
2017-02-15 16:47:01 -08:00
Noah Levitt
b409e49cfa deprecate current scope rule syntax and create new syntax with slightly different semantics (to be documented), and add parent_url_regex scope rule; unit test for scoping 2017-02-15 16:46:45 -08:00
Noah Levitt
f6fdb91d57 Merge branch 'master' into qa
* master:
  add --yaml option to brozzler-list-* commands
  take screenshot before running behavior (but after login) - thanks danielbicho
2017-02-15 23:13:32 +00:00
Noah Levitt
c0057e591a add --yaml option to brozzler-list-* commands 2017-02-15 23:13:09 +00:00
Noah Levitt
1054e8e3cb take screenshot before running behavior (but after login) - thanks danielbicho 2017-02-15 09:13:44 -08:00
Noah Levitt
08752a5163 Merge branch 'master' into qa
* master:
  logging tweaks
2017-02-10 15:19:35 -08:00
Noah Levitt
e58f4b7c44 logging tweaks 2017-02-10 15:19:28 -08:00
Barbara Miller
f990adfa63 post-merge tidying 2017-02-05 22:03:28 -08:00
Barbara Miller
46ee409de8 Merge branch 'simpledo' into qa 2017-02-05 22:01:07 -08:00
Barbara Miller
d4fdd089a3 finish jinja2 update 2017-02-05 21:58:54 -08:00
Barbara Miller
91205e6108 isVisible 2017-02-05 21:58:54 -08:00
Noah Levitt
01f5343b48 fix behaviors.yaml, switch simpledo behavior to jinja2 template 2017-02-05 21:58:54 -08:00
Barbara Miller
3851daa27d update for new behaviors set up 2017-02-05 21:58:54 -08:00
Barbara Miller
7296acdc19 simpledo first draft 2017-02-05 21:58:13 -08:00
Barbara Miller
1501d572da isVisible 2017-02-05 21:56:25 -08:00
Noah Levitt
98144a2180 fix behaviors.yaml, switch simpledo behavior to jinja2 template 2017-02-05 21:56:25 -08:00
Barbara Miller
dcc2dae8c1 update for new behaviors set up 2017-02-05 21:56:24 -08:00
Barbara Miller
c627d7ee84 simpledo first draft 2017-02-05 21:56:24 -08:00
Barbara Miller
0ff6257bef Merge branch 'qa' of github.com:internetarchive/brozzler into qa 2017-02-05 21:49:40 -08:00