Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-02-24 00:29:53 -05:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,492 Commits 69 Branches 15 Tags
Commit Graph

1492 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Barbara Miller
be330ba0c5 YTDLPStatus enum 2024-05-30 15:05:05 -07:00
Barbara Miller
9e64d2b7e8 fix typo 2024-05-30 11:18:20 -07:00
Barbara Miller
081ef320b9 add self._skip_av_seeds mostly 2024-05-30 11:10:06 -07:00
Barbara Miller
2e4a19e838 fixes for set, passed parameter 2024-05-30 10:13:05 -07:00
Barbara Miller
6fc409f554 update for default = False 2024-05-29 17:20:05 -07:00
Barbara Miller
a1a04aa0c5 cli.py updates from code review 2024-05-29 17:05:04 -07:00
Barbara Miller
4eb79ca936 add skip_ytdlp to class Site 2024-05-29 16:47:55 -07:00
Barbara Miller
a73059d25f black'd 2024-05-29 16:23:18 -07:00
Barbara Miller
2e560d3c18 skip_av_seeds 2024-05-29 16:11:18 -07:00
Barbara Miller
c66fec4c6d Merge remote-tracking branch 'upstream/master' into rotary_skip_ytdlp 2024-05-13 18:34:50 -07:00
Barbara Miller
d51d09c640
bump version 2024-04-29 14:46:43 -07:00
Barbara Miller
78120589fb
Merge pull request #278 from galgeek/nsw-pub 
behavior to capture opengov.nsw.gov document pages
 2024-04-29 14:44:38 -07:00
Barbara Miller
f22b000d0b opengov.nsw.gov behavior 2024-04-29 14:18:49 -07:00
Barbara Miller
5432844994 nsw.gov pubs behavior 2024-04-25 14:18:00 -07:00
Barbara Miller
ff02480a13 skip ytdlp for Rotary seeds 2024-04-24 17:25:49 -07:00
Barbara Miller
a7c2d469ca
bump version 2024-04-24 10:42:59 -07:00
Barbara Miller
68c132a2d3
Merge pull request #276 from galgeek/ytdlp_last 
run yt-dlp after brozzling a page, not before
 2024-04-24 10:41:37 -07:00
Barbara Miller
2d183c7d0c if not self._needs_browsing; _get_page_headers comment 2024-04-23 16:01:35 -07:00
Barbara Miller
7764c3f6d7 add comment 2024-04-23 16:00:18 -07:00
Barbara Miller
487a7009f0 rm remaining spy/fetch 2024-04-23 12:41:38 -07:00
Barbara Miller
5cc056cc7b use requests' CaseInsensitiveDict 2024-04-19 12:44:45 -07:00
Barbara Miller
12e49bf29e add / check status code for yt-dlp 2024-04-18 16:19:13 -07:00
Barbara Miller
423f05b841 black'd 2024-04-18 15:44:35 -07:00
Barbara Miller
700c80ba90 use page headers to browse or fetch 2024-04-18 15:41:00 -07:00
Barbara Miller
f2c89d1c18 skip more exts, plus chrome-error 2024-04-17 17:23:36 -07:00
Barbara Miller
737770e3ba locally black'd, to avoid github formatting error? 2024-04-17 10:48:03 -07:00
Vangelis Banos
7d580420ba Replace double quote with single quote 
Fix black warnings
 2024-04-15 19:23:44 +00:00
Vangelis Banos
1c0c385ff1 Use --headless=new or --headless=chrome if supported 
Ref: https://developer.chrome.com/docs/chromium/new-headless
 2024-04-15 18:48:54 +00:00
Barbara Miller
5fa25a747f mv append ytdlp_outlinks 2024-04-10 13:42:10 -07:00
Barbara Miller
0b26509637 more ytdlp_url 2024-04-10 13:41:03 -07:00
Barbara Miller
1bc9a544ef is_html_maybe 2024-04-04 12:24:45 -07:00
Barbara Miller
faa06b449d run yt-dlp after browse_page 2024-04-02 16:37:07 -07:00
Barbara Miller
376f3139ce
bump version 2024-03-13 11:46:55 -07:00
Barbara Miller
d72d8a622a
Merge pull request #273 from galgeek/eldo_agree_too 
add eldo.lu cookies dialog selector to defaults
 2024-03-13 11:44:12 -07:00
Barbara Miller
4b00249932 tighten rethinkdb dependency 2024-03-13 10:00:29 -07:00
Barbara Miller
7aab5d477f Merge branch 'eldo_agree' into eldo_agree_too 2024-03-13 09:50:49 -07:00
Barbara Miller
691e91eaed
bump version 2024-02-09 09:17:56 -08:00
Alex Dempsey
955cae6421
Merge pull request #271 from internetarchive/avdempsey/use-black 
Use black, enforce with GitHub Actions
 2024-02-08 22:35:27 -08:00
Alex Dempsey
8b23430a87 Use black, enforce with GitHub Actions 2024-02-08 12:07:41 -08:00
Barbara Miller
c4620c3018
bump version, copyright year 2024-02-05 13:00:14 -08:00
Barbara Miller
c8b45b889c more tweaks for default behavior 2024-02-05 12:32:00 -08:00
Barbara Miller
e9f63b5088 add default selector for didomi agree button 2024-02-05 12:08:53 -08:00
Barbara Miller
2f49b37076
Merge pull request #270 from internetarchive/moma-behavior 
MOMA behavior
 2024-02-02 13:52:26 -08:00
Barbara Miller
0270267136 fix indents 2024-01-20 13:00:05 -08:00
Barbara Miller
06a4ba10f7 refined 2024-01-19 14:41:45 -08:00
Barbara Miller
e2f572e968 initial moma behavior update 2023-12-20 15:25:03 -08:00
Barbara Miller
8afe9b5014
Merge pull request #268 from galgeek/skip_ytdlp4pdf 
@avdempsey approved elsewhere, for the record
 2023-12-12 11:39:11 -08:00
Barbara Miller
e422d5d23d bump version 2023-12-12 11:13:47 -08:00
Barbara Miller
2b0faec645 setup.py updates 2023-12-12 11:11:46 -08:00
Barbara Miller
88223cee59 skip .pdf 2023-12-11 17:25:19 -08:00