Drop federation transmission queues during a significant remote outage. (#7864)

* Empty federation transmission queues when we are backing off.

Fixes #7828.

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Address feedback

Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

* Reword newsfile
This commit is contained in:
reivilibre 2020-08-13 12:35:04 +01:00 committed by GitHub
parent dd8f28bd3f
commit ff0e894656
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 23 additions and 0 deletions

1
changelog.d/7864.bugfix Normal file
View File

@ -0,0 +1 @@
Fix a memory leak by limiting the length of time that messages will be queued for a remote server that has been unreachable.

View File

@ -337,6 +337,28 @@ class PerDestinationQueue(object):
(e.retry_last_ts + e.retry_interval) / 1000.0 (e.retry_last_ts + e.retry_interval) / 1000.0
), ),
) )
if e.retry_interval > 60 * 60 * 1000:
# we won't retry for another hour!
# (this suggests a significant outage)
# We drop pending PDUs and EDUs because otherwise they will
# rack up indefinitely.
# Note that:
# - the EDUs that are being dropped here are those that we can
# afford to drop (specifically, only typing notifications,
# read receipts and presence updates are being dropped here)
# - Other EDUs such as to_device messages are queued with a
# different mechanism
# - this is all volatile state that would be lost if the
# federation sender restarted anyway
# dropping read receipts is a bit sad but should be solved
# through another mechanism, because this is all volatile!
self._pending_pdus = []
self._pending_edus = []
self._pending_edus_keyed = {}
self._pending_presence = {}
self._pending_rrs = {}
except FederationDeniedError as e: except FederationDeniedError as e:
logger.info(e) logger.info(e)
except HttpResponseException as e: except HttpResponseException as e: