forked-synapse/synapse/storage
Eric Eastwood df8b91ed2b
Limit and filter the number of backfill points to get from the database (#13879)
There is no need to grab thousands of backfill points when we only need 5 to make the `/backfill` request with. We need to grab a few extra in case the first few aren't visible in the history.

Previously, we grabbed thousands of backfill points from the database, then sorted and filtered them in the app. Fetching the 4.6k backfill points for `#matrix:matrix.org` from the database takes ~50ms - ~570ms so it's not like this saves a lot of time 🤷. But it might save us more time now that `get_backfill_points_in_room`/`get_insertion_event_backward_extremities_in_room` are more complicated after https://github.com/matrix-org/synapse/pull/13635 

This PR moves the filtering and limiting to the SQL query so we just have less data to work with in the first place.

Part of https://github.com/matrix-org/synapse/issues/13356
2022-09-28 15:26:16 -05:00
..
controllers fix: Push notifications for invite over federation (#13719) 2022-09-28 12:31:53 +00:00
databases Limit and filter the number of backfill points to get from the database (#13879) 2022-09-28 15:26:16 -05:00
engines Snapshot schema 72 (#13873) 2022-09-26 18:28:32 +01:00
schema Handle remote device list updates during partial join (#13913) 2022-09-28 13:42:43 +00:00
util Cancel the processing of key query requests when they time out. (#13680) 2022-09-07 12:03:32 +01:00
__init__.py Rename storage classes (#12913) 2022-05-31 12:17:50 +00:00
_base.py Simplify cache invalidation after event persist txn (#13796) 2022-09-26 16:26:35 +01:00
background_updates.py Generate separate snapshots for logical databases (#13792) 2022-09-20 14:14:12 +01:00
database.py Snapshot schema 72 (#13873) 2022-09-26 18:28:32 +01:00
keys.py Use auto_attribs/native type hints for attrs classes. (#11692) 2022-01-13 13:49:28 +00:00
prepare_database.py Snapshot schema 72 (#13873) 2022-09-26 18:28:32 +01:00
push_rule.py Remove redundant "coding: utf-8" lines (#9786) 2021-04-14 15:34:27 +01:00
roommember.py Do not include rooms with an unknown room version in a sync response. (#10644) 2021-08-19 11:12:55 -04:00
state.py Update the rejected state of events during resync (#13459) 2022-08-11 10:42:24 +00:00
types.py Tidy up and type-hint the database engine modules (#12734) 2022-05-17 00:34:38 +01:00