synapse-product/synapse/storage/databases/main
Sean Quah 9cae44f49e
Track unconverted device list outbound pokes using a position instead (#14516)
When a local device list change is added to
`device_lists_changes_in_room`, the `converted_to_destinations` flag is
set to `FALSE` and the `_handle_new_device_update_async` background
process is started. This background process looks for unconverted rows
in `device_lists_changes_in_room`, copies them to
`device_lists_outbound_pokes` and updates the flag.

To update the `converted_to_destinations` flag, the database performs a
`DELETE` and `INSERT` internally, which fragments the table. To avoid
this, track unconverted rows using a `(stream ID, room ID)` position
instead of the flag.

From now on, the `converted_to_destinations` column indicates rows that
need converting to outbound pokes, but does not indicate whether the
conversion has already taken place.

Closes #14037.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-11-22 16:46:52 +00:00
..
__init__.py Merge/remove Slaved* stores into WorkerStores (#14375) 2022-11-11 10:51:49 +00:00
account_data.py Reintroduce #14376, with bugfix for monoliths (#14468) 2022-11-16 22:16:46 +00:00
appservice.py Check appservice user interest against the local users instead of all users (get_users_in_room mis-use) (#13958) 2022-10-27 18:29:23 +00:00
cache.py Batch fetch bundled references (#14508) 2022-11-22 09:41:09 -05:00
censor_events.py Safe async event cache (#13308) 2022-07-19 11:25:29 +00:00
client_ips.py Optimise _update_client_ips_batch_txn to batch together database operations. (#12252) 2022-04-08 15:29:13 +01:00
deviceinbox.py Add missing types to opentracing. (#13345) 2022-07-21 12:01:52 +00:00
devices.py Track unconverted device list outbound pokes using a position instead (#14516) 2022-11-22 16:46:52 +00:00
directory.py Replace uses of simple_insert_many with simple_insert_many_values. (#11742) 2022-01-13 19:44:18 -05:00
e2e_room_keys.py Remove redundant types from comments. (#14412) 2022-11-16 15:25:24 +00:00
end_to_end_keys.py Remove redundant types from comments. (#14412) 2022-11-16 15:25:24 +00:00
event_federation.py Stop getting missing prev_events after we already know their signature is invalid (#13816) 2022-10-15 00:36:49 -05:00
event_push_actions.py Update the thread_id right before use (in case the bg update hasn't finished) (#14222) 2022-10-18 14:55:41 +00:00
events_bg_updates.py Fix background update table-scanning events (#14374) 2022-11-07 14:28:00 +00:00
events_forward_extremities.py Fix returned count of delete extremities admin API (#12496) 2022-04-19 16:49:45 +01:00
events_worker.py Reintroduce #14376, with bugfix for monoliths (#14468) 2022-11-16 22:16:46 +00:00
events.py Batch fetch bundled references (#14508) 2022-11-22 09:41:09 -05:00
filtering.py Merge/remove Slaved* stores into WorkerStores (#14375) 2022-11-11 10:51:49 +00:00
keys.py Add some type hints to datastore (#12485) 2022-04-27 13:05:00 +01:00
lock.py Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
media_repository.py Replace noop background updates with DELETE. (#12954) 2022-06-13 14:06:27 -04:00
metrics.py Add some type hints to datastore (#12717) 2022-05-17 15:29:06 +01:00
monthly_active_users.py Remove redundant types from comments. (#14412) 2022-11-16 15:25:24 +00:00
openid.py Add type hints to some storage classes (#11307) 2021-11-11 08:47:31 -05:00
presence.py Revert "Make all process_replication_rows methods async (#13304)" (#13312) 2022-07-18 14:28:14 +01:00
profile.py Remove remaining pieces of groups code. (#12966) 2022-06-06 13:20:05 -04:00
purge_events.py Add support to purge rows from MSC2716 and other tables when purging a room (#13825) 2022-09-16 10:56:56 -05:00
push_rule.py Reintroduce #14376, with bugfix for monoliths (#14468) 2022-11-16 22:16:46 +00:00
pusher.py Reintroduce #14376, with bugfix for monoliths (#14468) 2022-11-16 22:16:46 +00:00
receipts.py Reintroduce #14376, with bugfix for monoliths (#14468) 2022-11-16 22:16:46 +00:00
registration.py Remove redundant types from comments. (#14412) 2022-11-16 15:25:24 +00:00
rejections.py Remove redundant "coding: utf-8" lines (#9786) 2021-04-14 15:34:27 +01:00
relations.py Batch fetch bundled references (#14508) 2022-11-22 09:41:09 -05:00
room_batch.py Correct type hint for room_batch.py (#11310) 2021-11-11 16:49:28 +00:00
room.py Remove redundant types from comments. (#14412) 2022-11-16 15:25:24 +00:00
roommember.py Include heroes in partial join responses' state (#14442) 2022-11-15 17:35:19 +00:00
search.py Drop support for Postgres 10 in full text search code. (#14397) 2022-11-09 09:55:34 -05:00
session.py Run pyupgrade --py37-plus --keep-percent-format on Synapse (#11685) 2022-01-05 09:53:05 -08:00
signatures.py remove constantly lib use and switch to enums. (#12624) 2022-05-04 11:26:11 +00:00
state_deltas.py Wait for lazy join to complete when getting current state (#12872) 2022-06-01 16:02:53 +01:00
state.py Instrument get_metadata_for_events for tracing (#13730) 2022-09-07 11:41:52 -05:00
stats.py Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
stream.py Merge/remove Slaved* stores into WorkerStores (#14375) 2022-11-11 10:51:49 +00:00
tags.py Revert "Make all process_replication_rows methods async (#13304)" (#13312) 2022-07-18 14:28:14 +01:00
transactions.py Require SQLite >= 3.27.0 (#13760) 2022-09-09 11:14:10 +01:00
ui_auth.py Add some type hints to datastore (#12485) 2022-04-27 13:05:00 +01:00
user_directory.py Remove redundant types from comments. (#14412) 2022-11-16 15:25:24 +00:00
user_erasure_store.py Annotations for user_erasure_store (#11313) 2021-11-11 19:22:19 +00:00