Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-02-24 08:39:59 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
558 Commits 69 Branches 15 Tags
Commit Graph

558 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Noah Levitt
f5e138c7eb Merge pull request #18 from ato/user-agent 
Add user_agent option
 2016-10-04 14:39:50 -07:00
Noah Levitt
692c1c48e1 Merge pull request #17 from ato/validate-conf 
Validate job conf against a simple schema
 2016-10-04 14:13:47 -07:00
Alex Osborne
eafc65938b Enforce at least one seed 2016-10-05 04:31:22 +11:00
Alex Osborne
743b5a4347 Add user_agent option 
Currently doesn't apply to requests made by youtube-dl as I
couldn't see a thread-safe way of doing that.
 2016-10-05 04:25:09 +11:00
Alex Osborne
5ac67fe513 Validate job conf against a Cerberus schema 2016-10-04 21:19:25 +11:00
Noah Levitt
15633be612 finish vagrant-brozzler-new-job.py 2016-10-03 18:17:35 -07:00
Noah Levitt
5bdf916a78 Merge pull request #16 from galgeek/logCaptchaHref 
log captcha, href
 2016-10-03 18:12:07 -07:00
Barbara Miller
c857647198 log location.href always 2016-10-03 17:37:45 -07:00
Barbara Miller
2f6ac120e3 log facebook captcha 2016-10-03 17:37:01 -07:00
Noah Levitt
de5c520ad7 pin psutil version, too 2016-10-03 17:12:00 -07:00
Noah Levitt
f220707aa4 pin pillow to version 3.3.0, primarily because we have a wheel in devpi for that version 2016-10-03 17:06:58 -07:00
Noah Levitt
df676223aa Merge branch 'master' of github.com:internetarchive/brozzler 
* 'master' of github.com:internetarchive/brozzler:
  Document how to use Flash with Headless Chromium
 2016-10-03 15:03:22 -07:00
Noah Levitt
0e096dd4e4 don't try to read the browser's cookie database if the browser hasn't been started (which can happen if the page is simply fetched rather than browsed because it's not html) 2016-10-03 15:03:08 -07:00
Noah Levitt
6de75717a4 Merge pull request #15 from ato/headless-flash 
Document how to use Flash with Headless Chromium
 2016-10-02 13:12:29 -07:00
Alex Osborne
a31ee2bcef Document how to use Flash with Headless Chromium 2016-10-02 23:40:41 +11:00
Noah Levitt
87a4d93a1c Merge pull request #13 from ato/headless 
Add notes on Headless Chromium to README
 2016-09-30 16:09:59 -07:00
Noah Levitt
bfd4c1f8c6 document a bunch of job settings 2016-09-29 16:15:44 -07:00
Noah Levitt
8c9a9c5666 starting on documenting job configuration 2016-09-29 12:03:16 -07:00
Alex Osborne
9062cb7b5b s/Chrome/Chromium/ 2016-09-29 08:03:27 +10:00
Alex Osborne
a4f908b22a Add notes on Headless Chromium 2016-09-29 07:59:25 +10:00
Noah Levitt
2462efc4ed replace vagrant-brozzler-new-site with python script that fills in default options and passes through others 2016-09-22 01:47:23 +01:00
Noah Levitt
cc9517cb45 add missing rethinkdb config file to ansible config 2016-09-22 01:45:28 +01:00
Noah Levitt
10d17ce7c3 Merge pull request #12 from galgeek/logFBlog 
better logs for facebook logins
 2016-09-19 21:18:17 -07:00
Noah Levitt
253122d061 new script runs brozzler-new-site queues a new site to brozzle on the vagrant brozzler deployment 2016-09-16 16:35:44 -07:00
Barbara Miller
8f44eac2f3 better logs for facebook logins 2016-09-16 15:42:19 -07:00
Noah Levitt
38af0f347b working on including pywb in vagrant environment (not finished) 2016-09-14 17:08:00 -07:00
Noah Levitt
c864499a64 starting to create a framework for testing 2016-09-14 17:06:49 -07:00
Noah Levitt
be27b4e16e header comment tweak 2016-09-14 17:04:32 -07:00
Noah Levitt
85073ab82b new prog "brozzler-wayback" runs monkey-patched pywb 2016-09-14 17:04:01 -07:00
Noah Levitt
1c5c9417d2 avoid "Uncaught TypeError: Cannot read property 'querySelectorAll' of undefined" from outlinks script 2016-08-25 13:10:30 -07:00
Noah Levitt
ed7e01210d little readme fix 2016-08-12 17:02:41 -07:00
Noah Levitt
c9bc9fb67d for vagrant, static ansible inventory file, add brozzler-webconsole 2016-08-10 18:41:23 -07:00
Noah Levitt
f671cf4f11 add info to display of jobless sites in brozzler-webconsole; fix creation of "least_hops" index on the rethinkdb table "pages" 2016-08-09 11:24:58 -07:00
Noah Levitt
74b229cfb0 add arguments --webconsole-address --webconsole-port --pywb-address and change default ports 2016-08-09 10:43:52 -07:00
Noah Levitt
94a8e70226 list jobless sites on brozzler-webconsole front page 2016-08-08 17:44:41 -07:00
Noah Levitt
4fa1571bc5 run brozzler-webconsole inside brozzler-easy 2016-08-08 17:43:38 -07:00
Noah Levitt
531b26aabb add section about brozzler-easy to the readme 2016-08-05 18:28:30 -07:00
Noah Levitt
c04bf85f4e add --help to brozzler-webconsole 2016-08-05 18:19:15 -07:00
Noah Levitt
ba6b342e28 fix exception happening now that we have binary data in rethinkdb (the cookie db) "TypeError: <binary, 7168 bytes, '53 51 4c 69 74 65...'> is not JSON serializable" 2016-08-05 17:12:22 -07:00
Noah Levitt
a211cc0514 dev version number again 2016-08-04 17:34:58 -07:00
Noah Levitt
ae63369c3c another version for pypi 1.1b5 2016-08-04 17:33:47 -07:00
Noah Levitt
20f9934dd9 avoid "Uncaught RangeError: Maximum call stack size exceeded" compiling outlinks 2016-08-04 17:33:06 -07:00
Noah Levitt
7734399a22 back to a dev version number 2016-08-04 16:00:42 -07:00
Noah Levitt
57c0d84fbd bump version to 1.1b4 for pypi upload 1.1b4 2016-08-04 15:55:56 -07:00
Noah Levitt
e62055d7d6 logging tweak 2016-08-04 15:54:05 -07:00
Noah Levitt
65d97caa9a install brozzler.webconsole package 2016-07-29 12:56:10 -05:00
Noah Levitt
cfc18e6845 add docstring to _chain_chrome_messages, remove debug logging, tweak name of websock thread 2016-07-28 20:29:11 -05:00
Noah Levitt
2046ee36e0 add a timeout to the one post-behavior step that didn't already have one (getting a screenshot), and majorly refactored the post-behavior code to incorporate timeouts automatically into each step, and hopefully make it easier to follow 2016-07-28 19:59:28 -05:00
Noah Levitt
b2b07b79a9 logging tweaks 2016-07-28 10:19:30 -05:00
Noah Levitt
dd2d8c89e3 reduce log level of messages from chrome, since it spews stuff that looks bad but usually isn't 2016-07-27 18:48:13 -05:00