Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
23,368 Commits 5 Branches 723 Tags 124 MiB
ab73abe693
Commit Graph

88 Commits

Author SHA1 Message Date
Patrick Cloke
2c6a7dfcbf
Use attempt_to_set_autocommit everywhere. (#16615) 
To avoid asserting the type of the database connection.
 2023-11-09 16:19:42 -05:00
Patrick Cloke
cfb6d38c47
Remove remaining usage of cursor_to_dict. (#16564) 2023-10-31 13:13:28 -04:00
Maxwell G
12611bfcdd
Add support for pydantic v2 via pydantic.v1 compat module (#16332) 
While maintaining support with pydantic v1.
 2023-09-25 15:19:08 +00:00
Patrick Cloke
aa483cb4c9
Update ruff config (#16283) 
Enable additional checks & clean-up unneeded configuration.
 2023-09-08 11:24:36 -04:00
David Robertson
6525fd65ee
Log the details of background update failures (#16212) 2023-09-01 12:41:56 +01:00
Shay
0377cb4fab
Override global statement timeout when creating indexes in Postgres (#16085) 2023-08-17 10:30:02 +01:00
Erik Johnston
e55a9b3e41
Fix downgrading to previous version of Synapse (#15907) 
We do this by marking the constraint as deferrable.
 2023-07-10 16:24:42 +01:00
Erik Johnston
95a96b21eb
Add foreign key constraint to event_forward_extremities. (#15751) 2023-07-05 09:43:19 +00:00
David Robertson
d162aecaac
Quick & dirty metric for background update status (#15740) 
* Quick & dirty metric for background update status

* Changelog

* Remove debug

Co-authored-by: Mathieu Velten <mathieuv@matrix.org>

* Actually write to _aborted

---------

Co-authored-by: Mathieu Velten <mathieuv@matrix.org>
 2023-06-07 17:12:23 +00:00
Patrick Cloke
2ad91ec628
Set thread_id column to non-null for event_push_{actions,actions_staging,summary} (#15597) 
Updates the database schema to require a thread_id (by adding a
constraint that the column is non-null) for event_push_actions,
event_push_actions_staging, and event_push_actions_summary.

For PostgreSQL we add the constraint as NOT VALID, then
VALIDATE the constraint a background job to avoid locking
the table during an upgrade.

Each table is updated as a separate schema delta to avoid
deadlocks between them.

For SQLite we simply rebuild the table & copy the data.
 2023-05-26 13:16:08 -04:00
Erik Johnston
808105bd31
Revert "Set thread_id column to non-null for event_push_{actions,actions_staging,summary} (#15437)" (#15580) 
This reverts commit a7b3e9ce65.
 2023-05-12 11:38:16 +01:00
Patrick Cloke
a7b3e9ce65
Set thread_id column to non-null for event_push_{actions,actions_staging,summary} (#15437) 
Updates the database schema to require a thread_id (by adding a
constraint that the column is non-null) for event_push_actions,
event_push_actions_staging, and event_push_actions_summary.

For PostgreSQL we add the constraint as NOT VALID, then
VALIDATE the constraint a background job to avoid locking
the table during an upgrade.

For SQLite we simply rebuild the table & copy the data.
 2023-05-03 07:49:03 -04:00
Sean Quah
373c485d8c
Handle half-created indices in receipts index background update (#14650) 
When Synapse is terminated while running the background update to create
the `receipts_graph` or `receipts_linearized` indexes, the indexes may
be successfully created (or marked as invalid on postgres) while the
background update remains unfinished. When Synapse next starts up, the
background update will fail because the index already exists, or exists
but is invalid on postgres.

Use the existing code to create indices in background updates, since it
handles these edge cases.

Signed-off-by: Sean Quah <seanq@matrix.org>
 2022-12-09 23:02:11 +00:00
David Robertson
fff9b955fa
Generate separate snapshots for logical databases (#13792) 
* Generate separate snapshots for sqlite, postgres and common
* Cleanup postgres dbs in the TRAP
* Say which logical DB we're applying updates to
* Run background updates on the state DB
* Add new option for accepting a SCHEMA_NUMBER
 2022-09-20 14:14:12 +01:00
Patrick Cloke
b2b0c85279
Support providing an index predicate for upserts. (#13822) 
This is useful to upsert against a table which has a unique
partial index while avoiding conflicts.
 2022-09-15 18:28:48 +00:00
Patrick Cloke
f2d12ccabe
Use partial indices on SQLIte. (#13802) 
Partial indices have been supported since SQLite 3.8, but Synapse
now requires >= 3.27, so we can enable support for them.

This requires rebuilding previous indices which were partial on
PostgreSQL, but not on SQLite.
 2022-09-14 12:01:42 -04:00
Patrick Cloke
53b77b203a
Replace noop background updates with DELETE. (#12954) 
Removes the `register_noop_background_update` and deletes the background
updates directly in a delta file.
 2022-06-13 14:06:27 -04:00
reivilibre
66a5f6c400
Add a unique index to state_group_edges to prevent duplicates being accidentally introduced and the consequential impact to performance. (#12687) 2022-05-19 14:16:49 +01:00
reivilibre
635f0d916b
Do not keep going if there are 5 back-to-back background update failures. (#12781) 2022-05-18 13:57:59 +00:00
David Robertson
aec69d2481
Another batch of type annotations (#12726) 2022-05-13 12:35:31 +01:00
Shay
ef3619e61d
Add config settings for background update parameters (#11980) 2022-03-11 10:46:45 -08:00
Shay
26211fec24
Fix a bug in background updates wherein background updates are never run using the default batch size (#12157) 2022-03-07 09:44:33 -08:00
Erik Johnston
d08ef6f155
Make background updates controllable via a plugin (#11306) 
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
 2021-11-29 17:57:06 +01:00
Brendan Abolivier
7f9841bdec
Lower minumum batch size to 1 for background updates (#11422) 
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2021-11-24 19:21:44 +00:00
Dirk Klimpel
ea20937084
Add an admin API to run background jobs. (#11352) 
Instead of having admins poke into the database directly.

Can currently run jobs to populate stats and to populate
the user directory.
 2021-11-19 19:39:46 +00:00
Erik Johnston
4ee71b9637
Add some background update admin APIs (#11263) 
Fixes #11259
 2021-11-08 16:08:02 +00:00
Jonathan de Jong
bdfde6dca1
Use inline type hints in http/federation/, storage/ and util/ (#10381) 2021-07-15 12:46:54 -04:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786) 
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
 2021-04-14 15:34:27 +01:00
Patrick Cloke
b7748d3c00
Import HomeServer from the proper module. (#9665) 2021-03-23 07:12:48 -04:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase (#9381) 
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
 2021-02-16 22:32:34 +00:00
Patrick Cloke
637282bb50
Add additional type hints to the storage module. (#8980) 2020-12-30 08:09:53 -05:00
Patrick Cloke
c619253db8
Stop sub-classing object (#8249) 2020-09-04 06:54:56 -04:00
Patrick Cloke
37db6252b7
Convert additional databases to async/await part 3 (#8201) 2020-09-01 11:04:17 -04:00
Patrick Cloke
30426c7063
Convert additional database methods to async (select list, search, insert_many, delete_*) (#8168) 2020-08-27 07:41:01 -04:00
Patrick Cloke
dbc630a628
Use the JSON encoder without whitespace in more places. (#8124) 2020-08-20 10:32:33 -04:00
Patrick Cloke
894dae74fe
Convert misc database code to async (#8087) 2020-08-14 07:24:26 -04:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense (#8033) 2020-08-05 21:38:57 +01:00
Patrick Cloke
f460da6031
Consistently use db_to_json to convert from database values to JSON objects. (#7849) 2020-07-16 11:32:19 -04:00
Richard van der Hoff
fcc2de7a0c Update docstring per review comments 2020-04-03 10:51:32 +01:00
Richard van der Hoff
af47264b78
review comment 2020-04-02 12:04:55 +01:00
Richard van der Hoff
7b608cf468 Only run one background update at a time 2020-03-31 17:43:58 +01:00
Richard van der Hoff
b4c2234232 Make do_next_background_update return a bool 
returning a None or an int that we don't use is confusing.
 2020-03-31 17:43:58 +01:00
Richard van der Hoff
26d17b9bdc Make has_completed_background_updates async 
(Almost) everywhere that uses it is happy with an awaitable.
 2020-03-31 17:43:58 +01:00
Richard van der Hoff
cfe8c8ab8e Remove unused start_background_update 
This was only used in a unit test, so let's just inline it in the test.
 2020-03-31 17:24:06 +01:00
Patrick Cloke
509e381afa
Clarify list/set/dict/tuple comprehensions and enforce via flake8 (#6957) 
Ensure good comprehension hygiene using flake8-comprehensions.
 2020-02-21 07:15:07 -05:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory (#6648) 
* Add a background update to clear tombstoned rooms from the directory

* use the ABC metaclass
 2020-01-07 14:18:43 +00:00
Richard van der Hoff
9824a39d80
Async/await for background updates (#6647) 
so that bg update routines can be async
 2020-01-07 14:12:42 +00:00
Erik Johnston
d537be1ebd Pass Database into the data store 2019-12-06 15:49:44 +00:00
Erik Johnston
4a33a6dd19 Move background update handling out of store 2019-12-05 11:11:26 +00:00
Erik Johnston
756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00