Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-02-24 08:39:59 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
724 Commits 69 Branches 15 Tags
Commit Graph

724 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Noah Levitt
24cc8377fb robots.txt for testing 2016-11-16 12:12:17 -08:00
Noah Levitt
3aead6de93 monkey-patch reppy to support substring user-agent matching 2016-11-16 11:41:34 -08:00
Noah Levitt
398871d46b give vagrant vm enough memory so that tests pass consistently 2016-11-14 18:26:00 -08:00
Noah Levitt
2b0a47c914 Merge pull request #27 from internetarchive/i2 
update Instagram behavior, mostly css selectors
 2016-11-14 12:40:55 -08:00
Noah Levitt
a74247412c need warcprox to listen on public address because that's what it puts in the service registry 2016-11-14 10:03:40 -08:00
Noah Levitt
c9b45a7e76 looks like the problem may have been a bug in ansible 2.2.0.0, so pin to 2.1.3.0 2016-11-14 09:58:13 -08:00
Barbara Miller
12a054e6dc update behavior, mostly css selectors 2016-11-14 09:20:40 -08:00
Noah Levitt
28b010a2ba back to dev version number 2016-11-11 14:58:55 -08:00
Noah Levitt
7aca046905 1.1b7 1.1b7 2016-11-11 14:58:07 -08:00
Noah Levitt
26b571219b use \n to delimit outlinks because urls can contain spaces (and anything else except [\n\t\0]) in the fragment part even after browser canonicalization 2016-11-11 14:14:47 -08:00
Noah Levitt
02bf23059e pass behavior_parameters from job configuration into Site objects 2016-11-09 13:43:10 -08:00
Noah Levitt
8e115b44fa add --behavior-parameters argument to brozzler-new-site 2016-11-09 13:12:36 -08:00
Noah Levitt
953e50d9a6 fix bug in final_bounces (not sure what I was thinking) 2016-11-09 13:12:14 -08:00
Noah Levitt
8889e4ab20 restore accidentally removed functionality handling page redirects and friends 2016-11-08 18:17:48 -08:00
Noah Levitt
054cb255ac cat logs on travis-ci failure 2016-11-08 14:26:12 -08:00
Noah Levitt
125a31165a reppy 0.4.1 has a significantly different api apparently, so for now let's go back to 0.3.4 2016-11-08 14:11:46 -08:00
Noah Levitt
fe18d915f5 still trying to get installation of pip to work on travis-ci 2016-11-08 13:50:12 -08:00
Noah Levitt
f10b4c71e6 update for reppy api change and pin to current version of reppy 2016-11-08 13:39:32 -08:00
Noah Levitt
cba5fa4a0b tweaks to ansible config to try to get the deployment to run on travis-ci 2016-11-08 13:31:52 -08:00
Noah Levitt
9d66f294ec move behavior_parameters into top level of site configuration 2016-11-07 18:16:04 -08:00
Noah Levitt
185d65bd5b Merge remote-tracking branch 'galgeek/login' 
* galgeek/login:
  add login details to behavior parameters
  initial login additions
 2016-11-07 18:15:43 -08:00
Noah Levitt
abca90a128 install the virtualenv package with pip because the apt version is old and conflicts with the recent version of pip we're using 2016-11-07 17:51:43 -08:00
Noah Levitt
99feeab581 logging tweak 2016-11-04 17:53:02 -07:00
Noah Levitt
5ac8994a24 rename webconsole to dashboard 2016-11-04 17:46:23 -07:00
Barbara Miller
c670bd060e add login details to behavior parameters 2016-11-02 16:51:19 -07:00
Barbara Miller
6c7f88c171 initial login additions 2016-11-02 16:04:18 -07:00
Noah Levitt
fef7d6a9fa Merge pull request #25 from ato/mouseovers-behavior 
Add a mouseovers behavior
 2016-10-31 11:55:46 -07:00
Noah Levitt
0c8ea52b08 Merge pull request #26 from ato/flash-doc 
Update Flash plugin instructions
 2016-10-31 11:55:02 -07:00
Alex Osborne
a1591a169a Update Flash plugin instructions 
libpepflashplayer.so is no longer included in the Chrome release
package.  Adobe are resuming Linux releases and the plugin is
now available from their download site.
 2016-10-29 14:12:33 +11:00
Noah Levitt
5bd4908e1d punycode host part of url to avoid errors doing WARCPROX_WRITE_RECORD 2016-10-26 13:50:23 -07:00
Noah Levitt
f30c143c66 avoid exception in case of url without host part 2016-10-26 12:45:24 -07:00
Noah Levitt
332912acd7 apparently response.status doesn't work sometimes; response.getcode() is documented so hopefully it keeps working 2016-10-25 17:50:49 -07:00
Noah Levitt
70ce642bee integer job ids are permitted as well as string 2016-10-21 21:25:16 +00:00
Alex Osborne
872030d716 Add a mouseovers behavior based on simpleclicks 2016-10-21 06:46:43 +11:00
Noah Levitt
21891476c4 avoid use of __double_underscore member variables because they're special https://shahriar.svbtle.com/underscores-in-python 2016-10-18 18:57:11 -07:00
Noah Levitt
becd832ea3 bump version after merging accept-encoding pull request 2016-10-18 17:55:00 -07:00
Noah Levitt
a146ba52ae Merge pull request #20 from internetarchive/rethinkInDoc 
rethinkdb installer works for me
 2016-10-18 17:54:48 -07:00
Noah Levitt
eedd2071bb Merge pull request #21 from galgeek/encodingFixIdentity 
ARI-5008 / ARI-5065 ExtraHTTPHeaders Accept-Encoding fix Identity
 2016-10-18 17:53:50 -07:00
Noah Levitt
aae34452f5 bump version number after merging travis-ci pull request 2016-10-18 17:48:45 -07:00
Noah Levitt
1490a40c28 Merge pull request #23 from internetarchive/travis 
Travis
 2016-10-18 17:48:04 -07:00
Noah Levitt
68a32fcbe2 bump version number after mouse's pull request 2016-10-18 17:45:55 -07:00
Noah Levitt
37950e6dfd Merge pull request #24 from mouse-reeve/select-proxy 
Use warcprox if enable_warcprox_features is true
 2016-10-18 17:41:56 -07:00
Mouse Reeve
2215aaab21 Use warcprox if enable_warcprox_features is true 2016-10-18 17:39:33 -07:00
Noah Levitt
62781117b8 tests passed on travis-ci, remove travis debugging lines 2016-10-18 17:31:47 -07:00
Noah Levitt
7a655da563 try to fix error from pip install with extras (pip install file:///...brozzler[webconsole]) by including #egg=brozzler as suggested at https://github.com/pypa/pip/issues/1236#issuecomment-32957253 , travis-ci error can be seen at https://travis-ci.org/internetarchive/brozzler/builds/168727451 2016-10-18 14:36:55 -07:00
Noah Levitt
0317409811 do ansible install as user travis to avoid permissions issues like this https://travis-ci.org/internetarchive/brozzler/builds/168720738 2016-10-18 12:44:18 -07:00
Noah Levitt
53624fd901 I don't know why travis/ansible/pip is failing, try using a file:// url https://travis-ci.org/internetarchive/brozzler/builds/168712667 2016-10-18 12:20:37 -07:00
Noah Levitt
166e87a00e trying to figure out this error from travis-ci 2016-10-18 11:50:55 -07:00
Noah Levitt
91c36f3d21 specify --cache-dir=/tmp/pip-cache to avoid trying to write to (nonexistent) service user home directory 2016-10-17 18:24:44 -07:00
Noah Levitt
0caf365516 travis-ci first try 2016-10-17 18:15:32 -07:00