Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-10-01 08:25:44 -04:00
Code Issues Packages Projects Releases Wiki Activity
f112cfe5bb
synapse-product/synapse/storage/databases/main
History
Erik Johnston f112cfe5bb
Fix MultiWriteIdGenerator's handling of restarts. (#8374) 
On startup `MultiWriteIdGenerator` fetches the maximum stream ID for
each instance from the table and uses that as its initial "current
position" for each writer. This is problematic as a) it involves either
a scan of events table or an index (neither of which is ideal), and b)
if rows are being persisted out of order elsewhere while the process
restarts then using the maximum stream ID is not correct. This could
theoretically lead to race conditions where e.g. events that are
persisted out of order are not sent down sync streams.

We fix this by creating a new table that tracks the current positions of
each writer to the stream, and update it each time we finish persisting
a new entry. This is a relatively small overhead when persisting events.
However for the cache invalidation stream this is a much bigger relative
overhead, so instead we note that for invalidation we don't actually
care about reliability over restarts (as there's no caches to
invalidate) and simply don't bother reading and writing to the new table
in that particular case.
2020-09-24 16:53:51 +01:00
..
schema Fix MultiWriteIdGenerator's handling of restarts. (#8374) 2020-09-24 16:53:51 +01:00
__init__.py Fix MultiWriteIdGenerator's handling of restarts. (#8374) 2020-09-24 16:53:51 +01:00
account_data.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
appservice.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
cache.py Separate get_current_token into two. (#8113) 2020-08-19 10:39:31 +01:00
censor_events.py Convert account data, device inbox, and censor events databases to async/await (#8063) 2020-08-12 09:29:06 -04:00
client_ips.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
deviceinbox.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
devices.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
directory.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
e2e_room_keys.py Convert additional database code to async/await. (#8195) 2020-08-28 07:54:27 -04:00
end_to_end_keys.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
event_federation.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
event_push_actions.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
events_bg_updates.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
events_worker.py Fix MultiWriteIdGenerator's handling of restarts. (#8374) 2020-09-24 16:53:51 +01:00
events.py Fix bug which caused failure on join with malformed membership events (#8385) 2020-09-23 16:42:14 +01:00
filtering.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
group_server.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
keys.py Convert additional database code to async/await. (#8195) 2020-08-28 07:54:27 -04:00
media_repository.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
metrics.py Convert tags and metrics databases to async/await (#8062) 2020-08-11 17:21:20 -04:00
monthly_active_users.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
openid.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
presence.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
profile.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
purge_events.py Catch-up after Federation Outage (split, 1) (#8230) 2020-09-04 12:22:23 +01:00
push_rule.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
pusher.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
receipts.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
registration.py Don't push if an user account has expired (#8353) 2020-09-23 16:06:28 +01:00
rejections.py Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
relations.py Convert additional databases to async/await (#8199) 2020-09-01 09:21:48 -04:00
room.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
roommember.py Add EventStreamPosition type (#8388) 2020-09-24 13:24:17 +01:00
search.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
signatures.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
state_deltas.py Convert state and stream stores and related code to async (#8194) 2020-08-28 09:37:55 -04:00
state.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
stats.py Fix missing null character check on guest_access room state 2020-09-22 19:39:29 +01:00
stream.py Synapse 1.20.0rc5 (2020-09-18) 2020-09-18 11:17:58 -04:00
tags.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
transactions.py Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322) 2020-09-18 14:59:13 +01:00
ui_auth.py Use slots in attrs classes where possible (#8296) 2020-09-14 12:50:06 -04:00
user_directory.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
user_erasure_store.py Fixed a bug with reactivating users with the admin API (#8362) 2020-09-22 18:19:01 +01:00