Erik Johnston
5829872bec
Fix port script to handle foreign key constraints ( #8730 )
2020-11-11 15:07:34 +00:00
Erik Johnston
1eb9de90c0
Improve start time by adding index to e2e_cross_signing_keys ( #8694 )
...
We do a `SELECT MAX(stream_id) FROM e2e_cross_signing_keys` on startup.
2020-11-02 13:55:56 +00:00
Andrew Morgan
a312e890f5
Cast errors generated during synapse_port_db to str ( #8585 )
...
I noticed in https://github.com/matrix-org/synapse/issues/8575 that the `end_error` variable in `synapse_port_db` is set to an `Exception`, even though later we expect it to be a `str`.
This PR simply casts an exception raised to a string. I'm doing this instead of having `end_error` be of type exception as we explicitly set `end_error` to a str here:
d25eb8f370/scripts/synapse_port_db (L542-L547)
2020-10-20 11:47:24 +01:00
Hubert Chathi
3cd78bbe9e
Add support for MSC2732: olm fallback keys ( #8312 )
2020-10-06 13:26:29 -04:00
Erik Johnston
e3debf9682
Add logging on startup/shutdown ( #8448 )
...
This is so we can tell what is going on when things are taking a while to start up.
The main change here is to ensure that transactions that are created during startup get correctly logged like normal transactions.
2020-10-02 15:20:45 +01:00
Dagfinn Ilmari Mannsåker
bd715e1278
Add ui_auth_sessions_ips table to synapse_port_db ignore list ( #8410 )
...
This table was created in #8034 (1.20.0). It references
`ui_auth_sessions`, which is ignored, so this one should be too.
Signed-off-by: Dagfinn Ilmari Mannsåker <ilmari@ilmari.org>
2020-09-28 15:35:02 +01:00
Erik Johnston
6fdf577593
Add new sequences to port DB script ( #8387 )
2020-09-24 13:43:49 +01:00
Patrick Cloke
13099ae431
Mark the shadow_banned column as boolean in synapse_port_db. ( #8386 )
2020-09-24 08:13:55 -04:00
Brendan Abolivier
2ffd6783c7
Revert #7736 ( #8039 )
2020-08-06 17:15:35 +01:00
Erik Johnston
a7bdf98d01
Rename database classes to make some sense ( #8033 )
2020-08-05 21:38:57 +01:00
Brendan Abolivier
8dff4a1242
Re-implement unread counts ( #7736 )
2020-07-29 18:26:55 +01:00
Richard van der Hoff
c445bc0cad
Use a postgres sequence to generate guest user IDs
2020-07-16 13:00:25 +01:00
Patrick Cloke
24110255cd
Sync ignored table names in synapse_port_db to current database schema ( #7717 )
2020-06-23 07:33:25 -04:00
Patrick Cloke
95e41f368b
Allow local media to be marked as safe from being quarantined. ( #7718 )
2020-06-22 08:04:14 -04:00
Patrick Cloke
6418b0379f
Ignore the UI Auth sessions when porting from sqlite to postgresql ( #7711 )
2020-06-17 10:01:18 -04:00
Dagfinn Ilmari Mannsåker
a3f11567d9
Replace all remaining six usage with native Python 3 equivalents ( #7704 )
2020-06-16 08:51:47 -04:00
Erik Johnston
e5c67d04db
Add option to move event persistence off master ( #7517 )
2020-05-22 16:11:35 +01:00
Erik Johnston
d7983b63a6
Support any process writing to cache invalidation stream. ( #7436 )
2020-05-07 13:51:08 +01:00
Brendan Abolivier
07124d028d
Port synapse_port_db to async/await ( #6718 )
...
* Raise an exception if there are pending background updates
So we return with a non-0 code
* Changelog
* Port synapse_port_db to async/await
* Port update_database to async/await
* Add version string to mocked homeservers
* Remove unused imports
* Convert overseen bits to async/await
* Fixup logging contexts
* Fix imports
* Add a way to print an error without raising an exception
* Incorporate review
2020-01-21 19:04:58 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership table ( #6655 )
...
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Richard van der Hoff
bf46821180
Refuse to start if sqlite is older than 3.11.0
2020-01-09 18:11:04 +00:00
Richard van der Hoff
e97d1cf001
Modify check_database to take a connection rather than a cursor
...
We might not need the cursor at all.
2020-01-09 18:05:50 +00:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory ( #6648 )
...
* Add a background update to clear tombstoned rooms from the directory
* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Erik Johnston
75d8f26ac8
Split state groups into a separate data store ( #6296 )
2019-12-20 10:48:24 +00:00
Erik Johnston
2284eb3a53
Add database config class ( #6513 )
...
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
Erik Johnston
71ee22c0ba
Fix port db script
2019-12-06 16:41:48 +00:00
Erik Johnston
8b77fc6506
Fix DB scripts
2019-12-05 11:20:49 +00:00
Erik Johnston
756d4942f5
Move DB pool and helper functions into dedicated Database class
2019-12-05 10:46:37 +00:00
Erik Johnston
ddbbfc9512
Merge pull request #6464 from matrix-org/erikj/make_public_sql_base
...
Clean up SQLBaseStore private function usage
2019-12-05 10:43:49 +00:00
Brendan Abolivier
f8421a1404
Fix background updates for synapse_port_db
2019-12-04 17:57:35 +00:00
Brendan Abolivier
6cd11109db
Make synapse_port_db exit with a non-0 code if something failed
2019-12-04 17:48:23 +00:00
Erik Johnston
ee86abb2d6
Remove underscore from SQLBaseStore functions
2019-12-04 16:23:43 +00:00
Andrew Morgan
57f09e01f5
Fix error when using synapse_port_db on a vanilla synapse db
2019-12-02 18:23:41 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase ( #6271 )
...
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Brendan Abolivier
0ef0b09d68
Merge pull request #6247 from matrix-org/babolivier/port_db_column
...
Add new boolean column to synapse_port_db
2019-10-24 17:07:34 +01:00
Andrew Morgan
ff39cb46de
Fix up changelog
2019-10-24 13:43:04 +01:00
Brendan Abolivier
2891693da6
Add new boolean column to synapse_port_db
2019-10-24 12:38:48 +01:00
Brendan Abolivier
b2510dce85
Fix import paths in synapse_port_db
2019-10-23 15:54:17 +01:00
Brendan Abolivier
c97ed64db3
Make synapse_port_db correctly create indexes ( #6102 )
...
Make `synapse_port_db` correctly create indexes in the PostgreSQL database, by having it run the background updates on the database before migrating the data.
To ensure we're migrating the right data, also block the port if the SQLite3 database still has pending or ongoing background updates.
Fixes #4877
2019-10-23 15:31:59 +01:00
Bart Noordervliet
dc4bec885d
Add missing BOOLEAN_COLUMNs to synapse_port_db ( #6216 )
...
Small fix to synapse_port_db to be able to convert from database schema v56.
2019-10-18 11:13:59 +02:00
Brendan Abolivier
83827c4922
Add account_validity's email_sent column to the list of boolean columns in synapse_port_db
...
Fixes #5306
2019-06-03 17:06:47 +01:00
Neil Johnson
2d951686a7
drop tables listed in #1830 ( #4992 )
...
Tables dropped:
* application_services,
* application_services_regex,
* transaction_id_to_pdu,
* stats_reporting
* current_state_resets
* event_content_hashes
* event_destinations
* event_edge_hashes
* event_signatures
* feedback
* room_hosts
* state_forward_extremities
2019-04-08 15:37:26 +01:00
Anders
2a59e8e429
Fix issue #4596
...
Make synapse_port_db --curses work with Python 3.
Signed-off-by: Anders Jensen-Waud <anders@jensenwaud.com>
2019-04-03 21:59:48 +00:00
Eric
8184ae8a09
Consider e2e_room_keys.is_verified column as boolean
...
This column was considered as an int, crashing the whole
migration process
Signed-off-by: Eric <eric@pedr0.net>
2019-02-20 23:18:00 +01:00
Richard van der Hoff
ef771cc4c2
Fix a number of flake8 errors
...
Broadly three things here:
* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
them
* use `r""` for strings which include backslashes
Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
Amber Brown
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) ( #4068 )
2018-10-20 11:16:55 +11:00
Adrian Tschira
e54c202b81
Replace some type checks with six type checks
...
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-07 01:02:32 +02:00
Richard van der Hoff
6a9777ba02
Port script: Set up state_group_id_seq
...
Fixes https://github.com/matrix-org/synapse/issues/3050 .
2018-04-06 15:33:30 +01:00
Richard van der Hoff
01579384cc
Port script: clean up a bit
...
Improve logging and comments. Group all the stuff to do with inspecting tables
together rather than creating the port tables in the middle.
2018-04-06 15:33:30 +01:00
Richard van der Hoff
e01ba5bda3
Port script: avoid nasty errors when setting up
...
We really shouldn't spit out "Failed to create port table", it looks scary.
2018-04-06 15:33:30 +01:00
