1574 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Barbara Miller	edf2f9e51a	retries for proxyrack and push	2024-09-11 09:52:32 -07:00
Adam Miller	0dca9f3861	chore: formatting	2024-08-30 21:15:59 +00:00
Adam Miller	18a976f82c	feat: Detect connection failures forwarded from warcprox and retry them with backoff	2024-08-30 21:11:02 +00:00
Adam Miller	0d8721a4d3	Bump version	2024-08-19 22:42:30 +00:00
Adam Miller	68a6953077	Merge pull request #284 from internetarchive/adam/adds_get_page_header_exception_handling ... fix: handle exceptions when requesting page headers for content-type …	2024-08-19 15:31:13 -07:00
Adam Miller	2dfe841493	fix: match original indenting	2024-08-12 23:53:04 +00:00
Adam Miller	f40bbd13d9	fix: handle exceptions when requesting page headers for content-type sampling.	2024-08-12 23:33:53 +00:00
Barbara Miller	43a37f02ce	bump version	2024-08-12 13:12:28 -07:00
Barbara Miller	fa24ee3685	Merge pull request #283 from galgeek/bmiller/WT-2862-yt-dlp-format-update ... prioritize video resolution — stop preferring single file download from yt-dlp	2024-08-12 12:42:09 -07:00
Barbara Miller	ef59f474bd	stop preferring single file download from ytdlp	2024-08-12 12:14:54 -07:00
Barbara Miller	16921d11be	bump version	2024-07-30 12:59:28 -07:00
Barbara Miller	c1f5aaf318	Merge pull request #282 from galgeek/2407-deps-update ... websocket-client update, plus yt-dlp update	2024-07-30 12:37:39 -07:00
Barbara Miller	124a390c11	Revert "fix doc typos" ... This reverts commit f06e0348eaf47f264698db3a8768ac981f311a28.	2024-07-29 14:28:53 -07:00
Barbara Miller	20841fb49d	mv yt-dlp to extras & update	2024-07-29 14:20:48 -07:00
Barbara Miller	81e640cd12	black'd	2024-07-29 13:47:11 -07:00
Barbara Miller	1ec4d0361a	skip websockt.enableTrace for now	2024-07-25 16:03:39 -07:00
Barbara Miller	704db5b957	update on_close for websocket-client update	2024-07-25 14:57:00 -07:00
Barbara Miller	f06e0348ea	fix doc typos	2024-07-23 17:58:04 -07:00
Barbara Miller	b41393fac5	yt_dlp 2024.7.16, websocket-client==1.8.0	2024-07-23 17:57:14 -07:00
Barbara Miller	4f36684c7c	Merge pull request #280 from galgeek/ytdlp_last_refined_really ... refine April ytdlp_last update	2024-07-18 15:10:11 -07:00
Barbara Miller	f1227ba74e	Merge branch 'master' into ytdlp_last_refined_really	2024-07-18 14:54:39 -07:00
Barbara Miller	34bd044c25	bump version	2024-07-18 10:27:33 -07:00
Barbara Miller	008b71ed6a	Merge pull request #274 from vbanos/headless-new ... Use --headless=new or --headless=chrome if supported	2024-07-18 10:26:30 -07:00
Barbara Miller	5a264f29d1	black'd	2024-07-18 10:19:44 -07:00
Barbara Miller	3f0227cd44	omit timestamp4datetime for now	2024-07-18 10:13:38 -07:00
Barbara Miller	a6d193171d	bump dev version	2024-07-17 17:32:24 -07:00
Barbara Miller	a0e1b1fe7e	refine ytdlp_last update, skipping adding attributes to page object	2024-07-17 17:31:21 -07:00
Barbara Miller	8d7507f377	bump version	2024-06-03 15:41:05 -07:00
Barbara Miller	42c5e6f559	Merge pull request #277 from galgeek/rotary_skip_ytdlp ... skip ytdlp for selected seeds	2024-06-03 15:19:31 -07:00
Barbara Miller	25f50c513e	skip unserializable enum YTDLPStatus	2024-06-03 13:43:15 -07:00
Barbara Miller	e4fa4ad6e5	black'd manually, again	2024-05-30 17:53:19 -07:00
Barbara Miller	90de4204b2	YTDLPStatus.SKIP not 'SKIP'	2024-05-30 16:05:56 -07:00
Barbara Miller	be330ba0c5	YTDLPStatus enum	2024-05-30 15:05:05 -07:00
Barbara Miller	9e64d2b7e8	fix typo	2024-05-30 11:18:20 -07:00
Barbara Miller	081ef320b9	add self._skip_av_seeds mostly	2024-05-30 11:10:06 -07:00
Barbara Miller	2e4a19e838	fixes for set, passed parameter	2024-05-30 10:13:05 -07:00
Barbara Miller	6fc409f554	update for default = False	2024-05-29 17:20:05 -07:00
Barbara Miller	a1a04aa0c5	cli.py updates from code review	2024-05-29 17:05:04 -07:00
Barbara Miller	4eb79ca936	add skip_ytdlp to class Site	2024-05-29 16:47:55 -07:00
Barbara Miller	a73059d25f	black'd	2024-05-29 16:23:18 -07:00
Barbara Miller	2e560d3c18	skip_av_seeds	2024-05-29 16:11:18 -07:00
Barbara Miller	c66fec4c6d	Merge remote-tracking branch 'upstream/master' into rotary_skip_ytdlp	2024-05-13 18:34:50 -07:00
Barbara Miller	d51d09c640	bump version	2024-04-29 14:46:43 -07:00
Barbara Miller	78120589fb	Merge pull request #278 from galgeek/nsw-pub ... behavior to capture opengov.nsw.gov document pages	2024-04-29 14:44:38 -07:00
Barbara Miller	f22b000d0b	opengov.nsw.gov behavior	2024-04-29 14:18:49 -07:00
Barbara Miller	5432844994	nsw.gov pubs behavior	2024-04-25 14:18:00 -07:00
Barbara Miller	ff02480a13	skip ytdlp for Rotary seeds	2024-04-24 17:25:49 -07:00
Barbara Miller	a7c2d469ca	bump version	2024-04-24 10:42:59 -07:00
Barbara Miller	68c132a2d3	Merge pull request #276 from galgeek/ytdlp_last ... run yt-dlp after brozzling a page, not before	2024-04-24 10:41:37 -07:00
Barbara Miller	2d183c7d0c	if not self._needs_browsing; _get_page_headers comment	2024-04-23 16:01:35 -07:00