Commit Graph

141 Commits

Author SHA1 Message Date
Erik Johnston
e5c67d04db
Add option to move event persistence off master (#7517) 2020-05-22 16:11:35 +01:00
Erik Johnston
d7983b63a6
Support any process writing to cache invalidation stream. (#7436) 2020-05-07 13:51:08 +01:00
Ivan Vilata-i-Balaguer
837f62266b Avoid attribute error when password_config present but empty (#6753)
The old statement returned `None` for such a `password_config` (like the one
created on first run), thus retrieval of the `pepper` key failed with
`AttributeError`.

Fixes #5315

Signed-off-by: Ivan Vilata i Balaguer <ivan@selidor.net>
2020-01-22 07:32:52 +00:00
Brendan Abolivier
07124d028d
Port synapse_port_db to async/await (#6718)
* Raise an exception if there are pending background updates

So we return with a non-0 code

* Changelog

* Port synapse_port_db to async/await

* Port update_database to async/await

* Add version string to mocked homeservers

* Remove unused imports

* Convert overseen bits to async/await

* Fixup logging contexts

* Fix imports

* Add a way to print an error without raising an exception

* Incorporate review
2020-01-21 19:04:58 +00:00
Erik Johnston
28c98e51ff
Add local_current_membership table (#6655)
Currently we rely on `current_state_events` to figure out what rooms a
user was in and their last membership event in there. However, if the
server leaves the room then the table may be cleaned up and that
information is lost. So lets add a table that separately holds that
information.
2020-01-15 14:59:33 +00:00
Richard van der Hoff
bf46821180 Refuse to start if sqlite is older than 3.11.0 2020-01-09 18:11:04 +00:00
Richard van der Hoff
e97d1cf001 Modify check_database to take a connection rather than a cursor
We might not need the cursor at all.
2020-01-09 18:05:50 +00:00
Richard van der Hoff
85db7f73be
Add a background update to clear tombstoned rooms from the directory (#6648)
* Add a background update to clear tombstoned rooms from the directory

* use the ABC metaclass
2020-01-07 14:18:43 +00:00
Richard van der Hoff
08815566bc
Automate generation of the sample and debian log configs (#6627) 2020-01-03 17:14:00 +00:00
Erik Johnston
75d8f26ac8
Split state groups into a separate data store (#6296) 2019-12-20 10:48:24 +00:00
Richard van der Hoff
b95b762560
Add an export_signing_key script (#6546)
I want to do some key rotation, and it is silly that we don't have a way to do
this.
2019-12-19 11:11:14 +00:00
Erik Johnston
2284eb3a53
Add database config class (#6513)
This encapsulates config for a given database and is the way to get new
connections.
2019-12-18 10:45:12 +00:00
Erik Johnston
71ee22c0ba Fix port db script 2019-12-06 16:41:48 +00:00
Erik Johnston
8b77fc6506 Fix DB scripts 2019-12-05 11:20:49 +00:00
Erik Johnston
756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Erik Johnston
ddbbfc9512
Merge pull request #6464 from matrix-org/erikj/make_public_sql_base
Clean up SQLBaseStore private function usage
2019-12-05 10:43:49 +00:00
Brendan Abolivier
f8421a1404
Fix background updates for synapse_port_db 2019-12-04 17:57:35 +00:00
Brendan Abolivier
6cd11109db
Make synapse_port_db exit with a non-0 code if something failed 2019-12-04 17:48:23 +00:00
Erik Johnston
ee86abb2d6 Remove underscore from SQLBaseStore functions 2019-12-04 16:23:43 +00:00
Andrew Morgan
57f09e01f5 Fix error when using synapse_port_db on a vanilla synapse db 2019-12-02 18:23:41 +00:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Brendan Abolivier
0ef0b09d68
Merge pull request #6247 from matrix-org/babolivier/port_db_column
Add new boolean column to synapse_port_db
2019-10-24 17:07:34 +01:00
Andrew Morgan
ff39cb46de Fix up changelog 2019-10-24 13:43:04 +01:00
Brendan Abolivier
2891693da6
Add new boolean column to synapse_port_db 2019-10-24 12:38:48 +01:00
Brendan Abolivier
b2510dce85
Fix import paths in synapse_port_db 2019-10-23 15:54:17 +01:00
Brendan Abolivier
c97ed64db3
Make synapse_port_db correctly create indexes (#6102)
Make `synapse_port_db` correctly create indexes in the PostgreSQL database, by having it run the background updates on the database before migrating the data.

To ensure we're migrating the right data, also block the port if the SQLite3 database still has pending or ongoing background updates.

Fixes #4877
2019-10-23 15:31:59 +01:00
Bart Noordervliet
dc4bec885d Add missing BOOLEAN_COLUMNs to synapse_port_db (#6216)
Small fix to synapse_port_db to be able to convert from database schema v56.
2019-10-18 11:13:59 +02:00
Amber Brown
be3b901ccd
Update the TLS cipher string and provide configurability for TLS on outgoing federation (#5550) 2019-06-28 18:19:09 +10:00
Amber Brown
32e7c9e7f2
Run Black. (#5482) 2019-06-20 19:32:02 +10:00
Amber Brown
6312d6cc7c
Expose statistics on extrems to prometheus (#5384) 2019-06-13 22:40:52 +10:00
Brendan Abolivier
843dd714cb
Merge pull request #5325 from matrix-org/babolivier/port_db_account_validity
Add account_validity's email_sent column to the list of boolean columns in synapse_port_db
2019-06-10 13:51:46 +01:00
Richard van der Hoff
fb98c05e03 add a script to generate new signing_key files 2019-06-05 15:33:02 +01:00
Brendan Abolivier
83827c4922
Add account_validity's email_sent column to the list of boolean columns in synapse_port_db
Fixes #5306
2019-06-03 17:06:47 +01:00
Neil Johnson
2d951686a7
drop tables listed in #1830 (#4992)
Tables dropped: 
 * application_services, 
 * application_services_regex, 
 * transaction_id_to_pdu, 
 * stats_reporting
 * current_state_resets
 * event_content_hashes
 * event_destinations
 * event_edge_hashes
 * event_signatures
 * feedback
 * room_hosts
 * state_forward_extremities
2019-04-08 15:37:26 +01:00
Anders
2a59e8e429 Fix issue #4596
Make synapse_port_db --curses work with Python 3.
Signed-off-by: Anders Jensen-Waud <anders@jensenwaud.com>
2019-04-03 21:59:48 +00:00
Richard van der Hoff
8e28bc5eee
Include a default configuration file in the 'docs' directory. (#4791) 2019-03-04 17:14:58 +00:00
Eric
8184ae8a09 Consider e2e_room_keys.is_verified column as boolean
This column was considered as an int, crashing the whole
migration process

Signed-off-by: Eric <eric@pedr0.net>
2019-02-20 23:18:00 +01:00
Richard van der Hoff
9c2af7b2c5 Add a script to generate a clean config file (#4315) 2018-12-22 02:04:57 +11:00
Amber Brown
264cb14402
Port hash_password to Python 3 (#4161)
* port hash_password

* changelog
2018-11-08 04:57:28 +11:00
Amber Brown
77d70a7646
Port register_new_matrix_user to Python 3 and add tests (#4085) 2018-10-26 22:05:22 +11:00
Richard van der Hoff
ef771cc4c2 Fix a number of flake8 errors
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
2018-10-24 10:39:03 +01:00
Amber Brown
e1728dfcbe
Make scripts/ and scripts-dev/ pass pyflakes (and the rest of the codebase on py3) (#4068) 2018-10-20 11:16:55 +11:00
Bruno Windels
9a8bbc9a59 add --no-admin flag to registration script (#3836) 2018-09-28 13:36:56 +01:00
Amber Brown
e1a237eaab
Admin API for creating new users (#3415) 2018-07-20 22:41:13 +10:00
Adrian Tschira
e54c202b81 Replace some type checks with six type checks
Signed-off-by: Adrian Tschira <nota@notafile.com>
2018-04-07 01:02:32 +02:00
Richard van der Hoff
6a9777ba02 Port script: Set up state_group_id_seq
Fixes https://github.com/matrix-org/synapse/issues/3050.
2018-04-06 15:33:30 +01:00
Richard van der Hoff
01579384cc Port script: clean up a bit
Improve logging and comments. Group all the stuff to do with inspecting tables
together rather than creating the port tables in the middle.
2018-04-06 15:33:30 +01:00
Richard van der Hoff
e01ba5bda3 Port script: avoid nasty errors when setting up
We really shouldn't spit out "Failed to create port table", it looks scary.
2018-04-06 15:33:30 +01:00
Richard van der Hoff
326189c25a Script to move remote media to another media store 2018-01-31 18:45:10 +00:00
Matthew Hodgson
3ab2cfec47 sanity checks 2017-11-21 12:10:20 +00:00