Matrix-Mirrors/synapse-product
1
0
Fork 0
mirror of https://git.anonymousland.org/anonymousland/synapse-product.git synced 2024-12-20 20:34:18 -05:00
Code Issues Packages Projects Releases Wiki Activity
52a50e8686
synapse-product/synapse/storage/databases/main
History
Erik Johnston 52a50e8686
Use vector clocks for room stream tokens. (#8439) 
Currently when using multiple event persisters we (in the worst case) don't tell clients about events until all event persisters have persisted new events after the original event. This is a suboptimal, especially if one of the event persisters goes down.

To handle this, we encode the position of each event persister in the room tokens so that we can send events to clients immediately. To reduce the size of the token we do two things:

1. We create a unique immutable persistent mapping between instance names and a generated small integer ID, which we can encode in the tokens instead of the instance name; and
2. We encode the "persisted upto position" of the room token and then only explicitly include instances that have positions strictly greater than that.

The new tokens look something like: `m3478~1.3488~2.3489`, where the first number is the min position, and the subsequent `-` separated pairs are the instance ID to positions map. (We use `.` and `~` as separators as they're URL safe and not already used by `StreamToken`).
2020-10-07 15:15:33 +01:00
..
schema Use vector clocks for room stream tokens. (#8439) 2020-10-07 15:15:33 +01:00
__init__.py Add logging on startup/shutdown (#8448) 2020-10-02 15:20:45 +01:00
account_data.py Do not assume that account data is of the correct form. (#8454) 2020-10-05 09:28:05 -04:00
appservice.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
cache.py Separate get_current_token into two. (#8113) 2020-08-19 10:39:31 +01:00
censor_events.py Enable mypy checking for unreachable code and fix instances. (#8432) 2020-10-01 08:09:18 -04:00
client_ips.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
deviceinbox.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
devices.py Add typing information to the device handler. (#8407) 2020-10-07 08:58:21 -04:00
directory.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
e2e_room_keys.py Convert additional database code to async/await. (#8195) 2020-08-28 07:54:27 -04:00
end_to_end_keys.py Add support for MSC2697: Dehydrated devices (#8380) 2020-10-07 08:00:17 -04:00
event_federation.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
event_push_actions.py Add logging on startup/shutdown (#8448) 2020-10-02 15:20:45 +01:00
events_bg_updates.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
events_worker.py Remove stream ordering from Metadata dict (#8452) 2020-10-05 14:43:14 +01:00
events.py Remove stream ordering from Metadata dict (#8452) 2020-10-05 14:43:14 +01:00
filtering.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
group_server.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
keys.py Convert additional database code to async/await. (#8195) 2020-08-28 07:54:27 -04:00
media_repository.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
metrics.py Allow background tasks to be run on a separate worker. (#8369) 2020-10-02 08:23:15 -04:00
monthly_active_users.py Add logging on startup/shutdown (#8448) 2020-10-02 15:20:45 +01:00
openid.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
presence.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
profile.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
purge_events.py Make token serializing/deserializing async (#8427) 2020-09-30 20:29:19 +01:00
push_rule.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
pusher.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
receipts.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
registration.py Add support for MSC2697: Dehydrated devices (#8380) 2020-10-07 08:00:17 -04:00
rejections.py Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
relations.py Convert additional databases to async/await (#8199) 2020-09-01 09:21:48 -04:00
room.py Allow background tasks to be run on a separate worker. (#8369) 2020-10-02 08:23:15 -04:00
roommember.py Add logging on startup/shutdown (#8448) 2020-10-02 15:20:45 +01:00
search.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
signatures.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
state_deltas.py Convert state and stream stores and related code to async (#8194) 2020-08-28 09:37:55 -04:00
state.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
stats.py Fix missing null character check on guest_access room state 2020-09-22 19:39:29 +01:00
stream.py Use vector clocks for room stream tokens. (#8439) 2020-10-07 15:15:33 +01:00
tags.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
transactions.py Catch-up after Federation Outage (bonus): Catch-up on Synapse Startup (#8322) 2020-09-18 14:59:13 +01:00
ui_auth.py Allow background tasks to be run on a separate worker. (#8369) 2020-10-02 08:23:15 -04:00
user_directory.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
user_erasure_store.py Fixed a bug with reactivating users with the admin API (#8362) 2020-09-22 18:19:01 +01:00