Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-20 23:56:34 -04:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

Adjust backoff timing to 60, 90, 135 depending on number of attempts

Browse Source
This commit is contained in:
Adam Miller 2024-11-12 13:07:07 -08:00
parent 0f100e6a71
commit ab488968c5
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
brozzler/worker.py
View File

@ -516,7 +516,7 @@ class BrozzlerWorker:
"unexpected exception site=%r page=%r", site, page, exc_info=True
)
if page:
retry_delay = min(60, 60 * (1.5**page.failed_attempts))
retry_delay = min(135, 60 * (1.5**page.failed_attempts))
page.retry_after = doublethink.utcnow() + datetime.timedelta(
seconds=retry_delay
)