synapse-product/synapse/storage/databases/main
Erik Johnston 19b15d63e8
Use autocommit mode for single statement DB functions. (#8542)
Autocommit means that we don't wrap the functions in transactions, and instead get executed directly. Introduced in #8456. This will help:

1. reduce the number of `could not serialize access due to concurrent delete` errors that we see (though there are a few functions that often cause serialization errors that we don't fix here);
2. improve the DB performance, as it no longer needs to deal with the overhead of `REPEATABLE READ` isolation levels; and
3. improve wall clock speed of these functions, as we no longer need to send `BEGIN` and `COMMIT` to the DB.

Some notes about the differences between autocommit mode and our default `REPEATABLE READ` transactions:

1. Currently `autocommit` only applies when using PostgreSQL, and is ignored when using SQLite (due to silliness with [Twisted DB classes](https://twistedmatrix.com/trac/ticket/9998)).
2. Autocommit functions may get retried on error, which means they can get applied *twice* (or more) to the DB (since they are not in a transaction the previous call would not get rolled back). This means that the functions need to be idempotent (or otherwise not care about being called multiple times). Read queries, simple deletes, and updates/upserts that replace rows (rather than generating new values from existing rows) are all idempotent.
3. Autocommit functions no longer get executed in [`REPEATABLE READ`](https://www.postgresql.org/docs/current/transaction-iso.html) isolation level, and so data can change queries, which is fine for single statement queries.
2020-10-14 15:50:59 +01:00
..
schema Fix message duplication if something goes wrong after persisting the event (#8476) 2020-10-13 12:07:56 +01:00
__init__.py Add logging on startup/shutdown (#8448) 2020-10-02 15:20:45 +01:00
account_data.py Do not assume that account data is of the correct form. (#8454) 2020-10-05 09:28:05 -04:00
appservice.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
cache.py Separate get_current_token into two. (#8113) 2020-08-19 10:39:31 +01:00
censor_events.py Move additional tasks to the background worker, part 3 (#8489) 2020-10-09 07:37:51 -04:00
client_ips.py Move additional tasks to the background worker (#8458) 2020-10-07 11:27:56 -04:00
deviceinbox.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
devices.py Move additional tasks to the background worker, part 3 (#8489) 2020-10-09 07:37:51 -04:00
directory.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
e2e_room_keys.py Convert additional database code to async/await. (#8195) 2020-08-28 07:54:27 -04:00
end_to_end_keys.py Invalidate the cache when an olm fallback key is uploaded (#8501) 2020-10-08 13:24:46 -04:00
event_federation.py Move additional tasks to the background worker, part 3 (#8489) 2020-10-09 07:37:51 -04:00
event_push_actions.py Move additional tasks to the background worker, part 3 (#8489) 2020-10-09 07:37:51 -04:00
events_bg_updates.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
events_worker.py Fix message duplication if something goes wrong after persisting the event (#8476) 2020-10-13 12:07:56 +01:00
events.py Make sure a retention policy is a state event (#8527) 2020-10-14 12:00:52 +01:00
filtering.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
group_server.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
keys.py Use autocommit mode for single statement DB functions. (#8542) 2020-10-14 15:50:59 +01:00
media_repository.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
metrics.py Move additional tasks to the background worker (#8458) 2020-10-07 11:27:56 -04:00
monthly_active_users.py Move additional tasks to the background worker, part 3 (#8489) 2020-10-09 07:37:51 -04:00
openid.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
presence.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
profile.py Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
purge_events.py Make token serializing/deserializing async (#8427) 2020-09-30 20:29:19 +01:00
push_rule.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
pusher.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
receipts.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
registration.py Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
rejections.py Convert simple_select_one and simple_select_one_onecol to async (#8162) 2020-08-26 07:19:32 -04:00
relations.py Convert additional databases to async/await (#8199) 2020-09-01 09:21:48 -04:00
room.py Move additional tasks to the background worker, part 4 (#8513) 2020-10-13 08:20:32 -04:00
roommember.py Move additional tasks to the background worker (#8458) 2020-10-07 11:27:56 -04:00
search.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
signatures.py Convert additional databases to async/await part 2 (#8200) 2020-09-01 08:39:04 -04:00
state_deltas.py Convert state and stream stores and related code to async (#8194) 2020-08-28 09:37:55 -04:00
state.py Simplify super() calls to Python 3 syntax. (#8344) 2020-09-18 09:56:44 -04:00
stats.py Fix missing null character check on guest_access room state 2020-09-22 19:39:29 +01:00
stream.py Use vector clocks for room stream tokens. (#8439) 2020-10-07 15:15:33 +01:00
tags.py Use async with for ID gens (#8383) 2020-09-23 16:11:18 +01:00
transactions.py Use autocommit mode for single statement DB functions. (#8542) 2020-10-14 15:50:59 +01:00
ui_auth.py Allow background tasks to be run on a separate worker. (#8369) 2020-10-02 08:23:15 -04:00
user_directory.py Use autocommit mode for single statement DB functions. (#8542) 2020-10-14 15:50:59 +01:00
user_erasure_store.py Fixed a bug with reactivating users with the admin API (#8362) 2020-09-22 18:19:01 +01:00