Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-08-10 23:40:30 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

Merge branch 'ARI-5744' into qa

Browse source
This commit is contained in:
Barbara Miller 2018-06-11 18:34:50 -07:00
commit 6b753623b7
2 changed files with 7 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
brozzler/behaviors.yaml
View file

@ -31,6 +31,12 @@
firstMatchOnly: true firstMatchOnly: true
- selector: a.coreSpriteRightPaginationArrow - selector: a.coreSpriteRightPaginationArrow
repeatSameElement: true repeatSameElement: true
-
url_regex: '^https?://(?:event\.)?crowdcompass\.com/.*$'
behavior_js_template: umbraBehavior.js.j2
default_parameters:
actions:
- selector: li.ember-view a
- -
url_regex: '^https?://americaspresidents\.si\.edu/gallery.*$' url_regex: '^https?://americaspresidents\.si\.edu/gallery.*$'
behavior_js_template: umbraBehavior.js.j2 behavior_js_template: umbraBehavior.js.j2

1
brozzler/worker.py
View file

@ -47,6 +47,7 @@ def _webpage_read_content(self, *args, **kwargs):
'bypassing youtube-dl extraction because content is ' 'bypassing youtube-dl extraction because content is '
'too large (%s characters)', len(content)) 'too large (%s characters)', len(content))
return '' return ''
return content
youtube_dl.extractor.generic.GenericIE._webpage_read_content = _webpage_read_content youtube_dl.extractor.generic.GenericIE._webpage_read_content = _webpage_read_content
class ExtraHeaderAdder(urllib.request.BaseHandler): class ExtraHeaderAdder(urllib.request.BaseHandler):