Git-Mirrors/brozzler
1
0
Fork 0
mirror of https://github.com/internetarchive/brozzler.git synced 2025-04-20 23:56:34 -04:00
Code Issues Actions 1 Packages Projects Releases Wiki Activity

chore: Comment retry delay backoff calculation.

Browse Source
This commit is contained in:
Adam Miller 2024-11-26 10:26:47 -08:00
parent 05ddc28ea8
commit 4b20d81b19
1 changed files with 2 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
brozzler/worker.py
View File

@ -552,6 +552,8 @@ class BrozzlerWorker:
"unexpected exception site=%r page=%r", site, page, exc_info=True
)
if page:
# Calculate backoff in seconds based on number of failed attempts.
# Minimum of 60, max of 135 giving delays of 60, 90, 135, 135...
retry_delay = min(135, 60 * (1.5**page.failed_attempts))
page.retry_after = doublethink.utcnow() + datetime.timedelta(
seconds=retry_delay