241 Commits

Author SHA1 Message Date
Noah Levitt
27452990ee toward getting initial tests to pass 2016-10-14 18:26:48 -07:00
Noah Levitt
5a373466a3 some vagrant/ansible fixes 2016-10-14 13:47:54 -07:00
Noah Levitt
3627209be1 move ansible directory to top level; generalize formerly vagrant-specific ansible configuration; let upstart manage logging with "console log" 2016-10-13 17:21:55 -07:00
Noah Levitt
56e651baeb working on basic integration tests 2016-10-13 17:12:35 -07:00
Noah Levitt
ed8b937277 back to dev version 2016-10-13 15:11:57 -07:00
Noah Levitt
456c082875 this is 1.1b6 2016-10-13 15:10:24 -07:00
Noah Levitt
23b59f8f4e Merge branch 'master' of github.com:internetarchive/brozzler
* 'master' of github.com:internetarchive/brozzler:
  Ensure job_schema.yaml is installed by pip
2016-10-13 15:00:13 -07:00
Noah Levitt
269512f499 make setup.py work with python2, not because the whole project works with python2, but just so it can be installed as a dependency of projects that support both python2 and python3 2016-10-13 14:59:31 -07:00
Noah Levitt
d82b40be68 fix typo 2016-10-06 18:20:41 -07:00
Alex Osborne
0fe2ef9387 Ensure job_schema.yaml is installed by pip
Use .yaml file extension for consistency with behaviors.yaml.
2016-10-05 15:45:11 +11:00
Noah Levitt
59a15d7f5c bump dev version after merging pull requests 2016-10-04 14:43:43 -07:00
Alex Osborne
5ac67fe513 Validate job conf against a Cerberus schema 2016-10-04 21:19:25 +11:00
Noah Levitt
15633be612 finish vagrant-brozzler-new-job.py 2016-10-03 18:17:35 -07:00
Noah Levitt
de5c520ad7 pin psutil version, too 2016-10-03 17:12:00 -07:00
Noah Levitt
f220707aa4 pin pillow to version 3.3.0, primarily because we have a wheel in devpi for that version 2016-10-03 17:06:58 -07:00
Noah Levitt
0e096dd4e4 don't try to read the browser's cookie database if the browser hasn't been started (which can happen if the page is simply fetched rather than browsed because it's not html) 2016-10-03 15:03:08 -07:00
Noah Levitt
bfd4c1f8c6 document a bunch of job settings 2016-09-29 16:15:44 -07:00
Noah Levitt
8c9a9c5666 starting on documenting job configuration 2016-09-29 12:03:16 -07:00
Noah Levitt
2462efc4ed replace vagrant-brozzler-new-site with python script that fills in default options and passes through others 2016-09-22 01:47:23 +01:00
Noah Levitt
cc9517cb45 add missing rethinkdb config file to ansible config 2016-09-22 01:45:28 +01:00
Noah Levitt
253122d061 new script runs brozzler-new-site queues a new site to brozzle on the vagrant brozzler deployment 2016-09-16 16:35:44 -07:00
Noah Levitt
38af0f347b working on including pywb in vagrant environment (not finished) 2016-09-14 17:08:00 -07:00
Noah Levitt
c864499a64 starting to create a framework for testing 2016-09-14 17:06:49 -07:00
Noah Levitt
be27b4e16e header comment tweak 2016-09-14 17:04:32 -07:00
Noah Levitt
85073ab82b new prog "brozzler-wayback" runs monkey-patched pywb 2016-09-14 17:04:01 -07:00
Noah Levitt
1c5c9417d2 avoid "Uncaught TypeError: Cannot read property 'querySelectorAll' of undefined" from outlinks script 2016-08-25 13:10:30 -07:00
Noah Levitt
ed7e01210d little readme fix 2016-08-12 17:02:41 -07:00
Noah Levitt
c9bc9fb67d for vagrant, static ansible inventory file, add brozzler-webconsole 2016-08-10 18:41:23 -07:00
Noah Levitt
f671cf4f11 add info to display of jobless sites in brozzler-webconsole; fix creation of "least_hops" index on the rethinkdb table "pages" 2016-08-09 11:24:58 -07:00
Noah Levitt
74b229cfb0 add arguments --webconsole-address --webconsole-port --pywb-address and change default ports 2016-08-09 10:43:52 -07:00
Noah Levitt
94a8e70226 list jobless sites on brozzler-webconsole front page 2016-08-08 17:44:41 -07:00
Noah Levitt
4fa1571bc5 run brozzler-webconsole inside brozzler-easy 2016-08-08 17:43:38 -07:00
Noah Levitt
531b26aabb add section about brozzler-easy to the readme 2016-08-05 18:28:30 -07:00
Noah Levitt
c04bf85f4e add --help to brozzler-webconsole 2016-08-05 18:19:15 -07:00
Noah Levitt
ba6b342e28 fix exception happening now that we have binary data in rethinkdb (the cookie db) "TypeError: <binary, 7168 bytes, '53 51 4c 69 74 65...'> is not JSON serializable" 2016-08-05 17:12:22 -07:00
Noah Levitt
a211cc0514 dev version number again 2016-08-04 17:34:58 -07:00
Noah Levitt
ae63369c3c another version for pypi 2016-08-04 17:33:47 -07:00
Noah Levitt
20f9934dd9 avoid "Uncaught RangeError: Maximum call stack size exceeded" compiling outlinks 2016-08-04 17:33:06 -07:00
Noah Levitt
7734399a22 back to a dev version number 2016-08-04 16:00:42 -07:00
Noah Levitt
57c0d84fbd bump version to 1.1b4 for pypi upload 2016-08-04 15:55:56 -07:00
Noah Levitt
e62055d7d6 logging tweak 2016-08-04 15:54:05 -07:00
Noah Levitt
65d97caa9a install brozzler.webconsole package 2016-07-29 12:56:10 -05:00
Noah Levitt
cfc18e6845 add docstring to _chain_chrome_messages, remove debug logging, tweak name of websock thread 2016-07-28 20:29:11 -05:00
Noah Levitt
2046ee36e0 add a timeout to the one post-behavior step that didn't already have one (getting a screenshot), and majorly refactored the post-behavior code to incorporate timeouts automatically into each step, and hopefully make it easier to follow 2016-07-28 19:59:28 -05:00
Noah Levitt
b2b07b79a9 logging tweaks 2016-07-28 10:19:30 -05:00
Noah Levitt
dd2d8c89e3 reduce log level of messages from chrome, since it spews stuff that looks bad but usually isn't 2016-07-27 18:48:13 -05:00
Noah Levitt
041a4970ce back to a dev version number 2016-07-27 16:57:42 -05:00
Noah Levitt
d94a7c23b9 1.1b3 for upload to pypi 2016-07-27 16:53:10 -05:00
Noah Levitt
c4bdb6c1fd pass behavior template parameters on to behavior - fixes umbra's ability to log in with parameters received from amqp 2016-07-26 19:47:09 -05:00
Noah Levitt
127002b77d brozzler[easy] requires warcprox>=2.0b1 2016-07-21 19:14:11 -05:00