Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-02-24 16:49:56 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
637 Commits 69 Branches 15 Tags
Commit Graph

637 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Noah Levitt
eedd2071bb Merge pull request #21 from galgeek/encodingFixIdentity 
ARI-5008 / ARI-5065 ExtraHTTPHeaders Accept-Encoding fix Identity
 2016-10-18 17:53:50 -07:00
Noah Levitt
aae34452f5 bump version number after merging travis-ci pull request 2016-10-18 17:48:45 -07:00
Noah Levitt
1490a40c28 Merge pull request #23 from internetarchive/travis 
Travis
 2016-10-18 17:48:04 -07:00
Noah Levitt
68a32fcbe2 bump version number after mouse's pull request 2016-10-18 17:45:55 -07:00
Noah Levitt
37950e6dfd Merge pull request #24 from mouse-reeve/select-proxy 
Use warcprox if enable_warcprox_features is true
 2016-10-18 17:41:56 -07:00
Mouse Reeve
2215aaab21 Use warcprox if enable_warcprox_features is true 2016-10-18 17:39:33 -07:00
Noah Levitt
62781117b8 tests passed on travis-ci, remove travis debugging lines 2016-10-18 17:31:47 -07:00
Noah Levitt
7a655da563 try to fix error from pip install with extras (pip install file:///...brozzler[webconsole]) by including #egg=brozzler as suggested at https://github.com/pypa/pip/issues/1236#issuecomment-32957253 , travis-ci error can be seen at https://travis-ci.org/internetarchive/brozzler/builds/168727451 2016-10-18 14:36:55 -07:00
Noah Levitt
0317409811 do ansible install as user travis to avoid permissions issues like this https://travis-ci.org/internetarchive/brozzler/builds/168720738 2016-10-18 12:44:18 -07:00
Noah Levitt
53624fd901 I don't know why travis/ansible/pip is failing, try using a file:// url https://travis-ci.org/internetarchive/brozzler/builds/168712667 2016-10-18 12:20:37 -07:00
Noah Levitt
166e87a00e trying to figure out this error from travis-ci 2016-10-18 11:50:55 -07:00
Noah Levitt
91c36f3d21 specify --cache-dir=/tmp/pip-cache to avoid trying to write to (nonexistent) service user home directory 2016-10-17 18:24:44 -07:00
Noah Levitt
0caf365516 travis-ci first try 2016-10-17 18:15:32 -07:00
Barbara Miller
0a6d8ed3da Accept-Encoding fix II 2016-10-17 12:50:31 -07:00
Barbara Miller
5ca4843a2e rethinkdb installer works for me 2016-10-16 11:01:12 -07:00
Noah Levitt
a370e7b987 tiny fix, and now the test passes for me 2016-10-14 19:21:26 -07:00
Noah Levitt
4044fcb647 fix pywb/brozzler replay of revisit records 2016-10-14 19:15:23 -07:00
Noah Levitt
27452990ee toward getting initial tests to pass 2016-10-14 18:26:48 -07:00
Noah Levitt
5a373466a3 some vagrant/ansible fixes 2016-10-14 13:47:54 -07:00
Noah Levitt
3627209be1 move ansible directory to top level; generalize formerly vagrant-specific ansible configuration; let upstart manage logging with "console log" 2016-10-13 17:21:55 -07:00
Noah Levitt
56e651baeb working on basic integration tests 2016-10-13 17:12:35 -07:00
Noah Levitt
ed8b937277 back to dev version 2016-10-13 15:11:57 -07:00
Noah Levitt
456c082875 this is 1.1b6 1.1b6 2016-10-13 15:10:24 -07:00
Noah Levitt
23b59f8f4e Merge branch 'master' of github.com:internetarchive/brozzler 
* 'master' of github.com:internetarchive/brozzler:
  Ensure job_schema.yaml is installed by pip
 2016-10-13 15:00:13 -07:00
Noah Levitt
269512f499 make setup.py work with python2, not because the whole project works with python2, but just so it can be installed as a dependency of projects that support both python2 and python3 2016-10-13 14:59:31 -07:00
Noah Levitt
d82b40be68 fix typo 2016-10-06 18:20:41 -07:00
Noah Levitt
c736f2e703 Merge pull request #19 from ato/schema-file-fix 
Ensure job_schema.yaml is installed by pip
 2016-10-05 09:02:18 -07:00
Alex Osborne
0fe2ef9387 Ensure job_schema.yaml is installed by pip 
Use .yaml file extension for consistency with behaviors.yaml.
 2016-10-05 15:45:11 +11:00
Noah Levitt
59a15d7f5c bump dev version after merging pull requests 2016-10-04 14:43:43 -07:00
Noah Levitt
f5e138c7eb Merge pull request #18 from ato/user-agent 
Add user_agent option
 2016-10-04 14:39:50 -07:00
Noah Levitt
692c1c48e1 Merge pull request #17 from ato/validate-conf 
Validate job conf against a simple schema
 2016-10-04 14:13:47 -07:00
Alex Osborne
eafc65938b Enforce at least one seed 2016-10-05 04:31:22 +11:00
Alex Osborne
743b5a4347 Add user_agent option 
Currently doesn't apply to requests made by youtube-dl as I
couldn't see a thread-safe way of doing that.
 2016-10-05 04:25:09 +11:00
Alex Osborne
5ac67fe513 Validate job conf against a Cerberus schema 2016-10-04 21:19:25 +11:00
Noah Levitt
15633be612 finish vagrant-brozzler-new-job.py 2016-10-03 18:17:35 -07:00
Noah Levitt
5bdf916a78 Merge pull request #16 from galgeek/logCaptchaHref 
log captcha, href
 2016-10-03 18:12:07 -07:00
Barbara Miller
c857647198 log location.href always 2016-10-03 17:37:45 -07:00
Barbara Miller
2f6ac120e3 log facebook captcha 2016-10-03 17:37:01 -07:00
Noah Levitt
de5c520ad7 pin psutil version, too 2016-10-03 17:12:00 -07:00
Noah Levitt
f220707aa4 pin pillow to version 3.3.0, primarily because we have a wheel in devpi for that version 2016-10-03 17:06:58 -07:00
Noah Levitt
df676223aa Merge branch 'master' of github.com:internetarchive/brozzler 
* 'master' of github.com:internetarchive/brozzler:
  Document how to use Flash with Headless Chromium
 2016-10-03 15:03:22 -07:00
Noah Levitt
0e096dd4e4 don't try to read the browser's cookie database if the browser hasn't been started (which can happen if the page is simply fetched rather than browsed because it's not html) 2016-10-03 15:03:08 -07:00
Noah Levitt
6de75717a4 Merge pull request #15 from ato/headless-flash 
Document how to use Flash with Headless Chromium
 2016-10-02 13:12:29 -07:00
Alex Osborne
a31ee2bcef Document how to use Flash with Headless Chromium 2016-10-02 23:40:41 +11:00
Noah Levitt
87a4d93a1c Merge pull request #13 from ato/headless 
Add notes on Headless Chromium to README
 2016-09-30 16:09:59 -07:00
Noah Levitt
bfd4c1f8c6 document a bunch of job settings 2016-09-29 16:15:44 -07:00
Noah Levitt
8c9a9c5666 starting on documenting job configuration 2016-09-29 12:03:16 -07:00
Alex Osborne
9062cb7b5b s/Chrome/Chromium/ 2016-09-29 08:03:27 +10:00
Alex Osborne
a4f908b22a Add notes on Headless Chromium 2016-09-29 07:59:25 +10:00
Noah Levitt
2462efc4ed replace vagrant-brozzler-new-site with python script that fills in default options and passes through others 2016-09-22 01:47:23 +01:00