Add back fast path for non-gappy syncs (#17064)

PR #16942 removed an invalid optimisation that avoided pulling out state
for non-gappy syncs. This causes a large increase in DB usage. c.f.
#16941 for why that optimisation was wrong.

However, we can still optimise in the simple case where the events in
the timeline are a linear chain without any branching/merging of the
DAG.

cc. @richvdh
This commit is contained in:
Erik Johnston 2024-04-08 14:25:28 +01:00 committed by GitHub
parent 5360baeb64
commit 1f8f991d51
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
2 changed files with 37 additions and 0 deletions

1
changelog.d/17064.bugfix Normal file
View File

@ -0,0 +1 @@
Fix various long-standing bugs which could cause incorrect state to be returned from `/sync` in certain situations.

View File

@ -1259,6 +1259,42 @@ class SyncHandler:
await_full_state = True await_full_state = True
lazy_load_members = False lazy_load_members = False
# For a non-gappy sync if the events in the timeline are simply a linear
# chain (i.e. no merging/branching of the graph), then we know the state
# delta between the end of the previous sync and start of the new one is
# empty.
#
# c.f. #16941 for an example of why we can't do this for all non-gappy
# syncs.
is_linear_timeline = False
if batch.events:
prev_event_id = batch.events[0].event_id
for e in batch.events[1:]:
if e.prev_event_ids() != [prev_event_id]:
break
else:
is_linear_timeline = True
if is_linear_timeline and not batch.limited:
state_ids: StateMap[str] = {}
if lazy_load_members:
if members_to_fetch and batch.events:
# We're lazy-loading, so the client might need some more
# member events to understand the events in this timeline.
# So we fish out all the member events corresponding to the
# timeline here. The caller will then dedupe any redundant
# ones.
state_ids = await self._state_storage_controller.get_state_ids_for_event(
batch.events[0].event_id,
# we only want members!
state_filter=StateFilter.from_types(
(EventTypes.Member, member) for member in members_to_fetch
),
await_full_state=False,
)
return state_ids
if batch: if batch:
state_at_timeline_start = ( state_at_timeline_start = (
await self._state_storage_controller.get_state_ids_for_event( await self._state_storage_controller.get_state_ids_for_event(