Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-09-22 13:44:51 -04:00
Code Issues Projects Releases Packages Wiki Activity Actions

log urls we skipped adding to outlinks

Browse source
This commit is contained in:
Barbara Miller 2025-07-15 17:16:00 -07:00
parent 3af8247277
commit 5b967b9738
1 changed files with 1 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
brozzler/ydl.py
View file

@ -619,6 +619,7 @@ def do_youtube_dl(worker, site, page, ytdlp_proxy_endpoints):
urlcanon.aggressive(f"http://www.youtube.com/watch?v={e['id']}")
)
if youtube_watch_url in captured_youtube_watch_pages:
logger.info("skipping adding %s to outlinks", youtube_watch_url)
continue
uncaptured_youtube_watch_pages.append(
f"https://www.youtube.com/watch?v={e['id']}"