Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-22 16:39:07 -04:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity
1,534 Commits 63 Branches 16 Tags
Commit Graph

1534 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Barbara Miller
24fcca4919 Merge branch 'pageInterstitialShown' into qa 2018-09-27 16:31:28 -07:00
Barbara Miller
1054d2d644 return outlinks = [] 2018-09-27 16:22:12 -07:00
Barbara Miller
0ed131ce72 simplest exception ok 2018-09-27 16:19:08 -07:00
Barbara Miller
60cfd684b2 Merge branch 'pageInterstitialShown' into qa 2018-09-25 10:30:02 -07:00
Barbara Miller
d9f7997a40 except log and return [] 2018-09-24 20:23:46 -07:00
Barbara Miller
156ec0caa1 tidier, better exception handling? 2018-09-21 17:46:19 -07:00
Barbara Miller
51d011c285 expect exception PageInterstitialShown 2018-09-20 17:23:49 -07:00
Barbara Miller
86193a525b raise exception PageInterstitialShown 2018-09-20 17:22:52 -07:00
Barbara Miller
66936c67cc add exception PageIntersitialShown 2018-09-20 17:21:49 -07:00
Barbara Miller
0e867102a9 Browsing Exception for Page.interstitialShown 2018-09-20 15:41:31 -07:00
Noah Levitt
48bf185746 bump version after merge 2018-09-18 11:08:44 -07:00
Noah Levitt
dceee8bdbd
Merge pull request #119 from nlevitt/ydl-stitch-fix 
WIP youtube-dl stitching fixes
 2018-09-18 11:08:21 -07:00
Noah Levitt
60cd69e2bd send warcprox-meta when pushing stitched up video 
also put locking around monkey patching to avoid race condition
 2018-09-18 01:07:52 -07:00
Noah Levitt
1ef717fa75 test exposing bug that we don't send warcprox-meta 
when pushing stitched-up video with WARCPROX_WRITE_RECORD
 2018-09-18 01:05:18 -07:00
Neil Minton
3c7fdeae2c Merge branch 'ari-5777' into qa 2018-09-12 12:07:45 -04:00
Neil Minton
b5c213cef9 Extract srcset values for use in crawling. 2018-09-12 12:04:47 -04:00
Noah Levitt
efb0696833 bump version number after merge 2018-09-06 16:17:59 -07:00
jkafader
8368cd2bcb
Merge pull request #115 from nlevitt/ydl-stitched 
Ydl stitched
 2018-09-06 16:15:52 -07:00
Noah Levitt
c4fdbe578d Merge branch 'master' into qa 
* master:
  oops, back to dev version number
  wait 20 seconds to claim sites if none were avail-
  tweak logging
  why did those tests fail??? (#117)
  Add screenshots
  Add screenshots
  back to dev version
  1.4 for pypi
  explain --warcprox-auto briefly
  vagrant readme fixes (thanks funkyfuture)
  update cryptography dep version
 2018-09-04 10:54:26 -07:00
Noah Levitt
a4eacb5b8f oops, back to dev version number 2018-09-04 10:52:34 -07:00
jkafader
e38b867ff5
Merge pull request #118 from nlevitt/relax-claiming 
Relax claiming
 2018-09-04 10:45:11 -07:00
Noah Levitt
2d5c6681cf wait 20 seconds to claim sites if none were avail- 
able last time, up from 0.5 seconds
this should lighten the load on rethinkdb considerably
 2018-08-31 15:23:59 -07:00
Noah Levitt
d0f5cd7168 tweak logging 2018-08-31 15:23:48 -07:00
Noah Levitt
88d3d3b310
why did those tests fail??? (#117) 
1.4 for pypi
1.4 		2018-08-22 14:35:39 -07:00
Noah Levitt
02e98f101d
Merge pull request #116 from kblumenthal/master 
Add screenshots
 2018-08-22 14:34:52 -07:00
Karl-Rainer Blumenthal
ff1645ef7d
Add screenshots 
Add Brozzler Dashboard and Wayback screenshots to readme
 2018-08-22 13:02:08 -04:00
Karl-Rainer Blumenthal
7c8b597ad3
Add screenshots 
Add screenshots of Brozzler Dashboard and Wayback
 2018-08-22 12:55:10 -04:00
Noah Levitt
2a2952e810 back to dev version 2018-08-21 15:18:18 -07:00
Noah Levitt
b63661ea70 1.4 for pypi 2018-08-21 15:15:38 -07:00
Noah Levitt
eaf7ef74be explain --warcprox-auto briefly 2018-08-17 12:06:04 -07:00
Karl-Rainer Blumenthal
2081e6388a
Merge pull request #2 from internetarchive/master 
Updating to upstream origin
 2018-08-17 14:26:46 -04:00
Noah Levitt
8cdc3dee21 Merge branch 'master' into ydl-stitched 
* master:
  vagrant readme fixes (thanks funkyfuture)
  update cryptography dep version
 2018-08-17 10:34:00 -07:00
Noah Levitt
d19e139101 vagrant readme fixes (thanks funkyfuture) 2018-08-17 10:31:01 -07:00
Noah Levitt
ffa8021968 update cryptography dep version 
github tells me there's a vulnerability <2.3
 2018-08-16 14:32:03 -07:00
Noah Levitt
ecb2d70369 youtube-dl: skip youtube dash manifests 
missed in last merge commit
replay of 6a4bcaca1b
 2018-08-16 14:25:37 -07:00
Noah Levitt
cbeba3a6b9 Merge branch 'ydl-stitched' into qa 
* ydl-stitched:
  fix failing tests
  test for youtube-dl stitch-up
  add missing imports and fix mimetype issue
  move youtube-dl code into separate file
  push youtube-dl's stitched up videos to warcprox
 2018-08-16 12:10:44 -07:00
Noah Levitt
418a3ef20c Merge branch 'master' into qa 
* master:
  expose more brozzle-page args
  update pillow dependency to get rid of github vul-
  more readme edits
  reformat readme to 80 columns
  Copy edits to job-conf readme
  bump up heartbeat interval (see comment)
  Copy edits
  back to dev version
  version 1.3 (messed up 1.2)
  setuptools wants README not readme
  back to dev version number
  version 1.2
  bump dev version after merge
  is test_time_limit is failing because of timing?
 2018-08-16 12:08:48 -07:00
Noah Levitt
e7d2273856 fix failing tests 2018-08-16 11:40:54 -07:00
Noah Levitt
3c27132aaa test for youtube-dl stitch-up 2018-08-15 17:42:53 -07:00
Noah Levitt
c2ad8427e1 add missing imports and fix mimetype issue 2018-08-15 17:41:35 -07:00
Noah Levitt
33520da8f9 move youtube-dl code into separate file 2018-08-14 15:10:48 -07:00
Noah Levitt
39155ebcc5 push youtube-dl's stitched up videos to warcprox 
(no tests yet)
 2018-08-13 15:40:48 -07:00
Noah Levitt
4e398e1da2 expose more brozzle-page args 2018-08-13 15:38:24 -07:00
Noah Levitt
b44a444dc2 update pillow dependency to get rid of github vul- 
nerability warning
 2018-07-24 16:37:25 -05:00
Noah Levitt
771d6aa626 more readme edits 2018-07-23 19:05:49 -05:00
Noah Levitt
073fc713f4
Merge pull request #113 from nlevitt/karl-readme 
Karl readme copy edits
 2018-07-23 18:36:00 -05:00
Noah Levitt
f7407a87c1 reformat readme to 80 columns 2018-07-23 23:32:56 +00:00
Noah Levitt
a7fb7bcc37 Merge branch 'master' into karl 
* master:
  bump up heartbeat interval (see comment)
  back to dev version
  version 1.3 (messed up 1.2)
  setuptools wants README not readme
  back to dev version number
  version 1.2
  bump dev version after merge
  is test_time_limit is failing because of timing?
  fix bug in test, add another one
  treat any error fetching robots.txt as "allow all"
  update instagram behavior
 2018-07-23 23:28:42 +00:00
Karl-Rainer Blumenthal
bd78e07232
Copy edits to job-conf readme 
Good reading and rampant pedantry!
 2018-07-06 15:24:12 -04:00
Noah Levitt
9d18dc6aeb bump up heartbeat interval (see comment) 2018-07-03 18:35:08 -05:00