Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
15,854 Commits 5 Branches 723 Tags 124 MiB
72acca6a32
Commit Graph

11145 Commits

Author SHA1 Message Date
Erik Johnston
72acca6a32 Back out change preventing setting null avatar URLs 2019-12-11 11:46:55 +00:00
Erik Johnston
d21577bdcb Merge branch 'erikj/fix_sqlite_7' of github.com:matrix-org/synapse into release-v1.7.0 2019-12-11 11:34:50 +00:00
Brendan Abolivier
3bd049bbb7
Give the server config to the RoomWorkerStore 2019-12-10 13:05:35 +00:00
Erik Johnston
52346990c8 Drop unused index 2019-12-10 12:46:14 +00:00
Erik Johnston
31da85e467 Convert _censor_redactions to async since it awaits on coroutines 2019-12-10 12:46:00 +00:00
Brendan Abolivier
2ac78438d8
Make the PusherSlaveStore inherit from the slave RoomStore 
So that it has access to the get_retention_policy_for_room function which is required by filter_events_for_client.
 2019-12-10 12:31:03 +00:00
Erik Johnston
cc5f6eb608 Only start censor background job after indices are created 2019-12-10 11:39:31 +00:00
Erik Johnston
4cade96616 Fix support for SQLite 3.7. 
Partial indices support was added in 3.8.0, so we need to use the
background updates that handles this correctly.
 2019-12-09 15:09:16 +00:00
Neil Johnson
0f3614f0f6 Synapse 1.7.0rc1 (2019-12-09) 
=============================
 
 Features
 --------
 
 - Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436))
 - Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858))
 - Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925))
 - Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119))
 - Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176))
 - Configure privacy-preserving settings by default for the room directory. ([\#6354](https://github.com/matrix-org/synapse/issues/6354))
 - Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409))
 - Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434))
 
 Bugfixes
 --------
 
 - Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237))
 - Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241))
 - Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329))
 - Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332))
 - Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333))
 - Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406))
 - Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408))
 - Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420))
 - Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421))
 - Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449))
 - Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451))
 - Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462))
 - Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470))
 - Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472))
 - Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491))
 - Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493))
 - Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494))
 
 Improved Documentation
 ----------------------
 
 - Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490))
 - Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388))
 - Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390))
 - Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443))
 - Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458))
 - Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461))
 
 Deprecations and Removals
 -------------------------
 
 - Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925))
 - Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488))
 
 Internal Changes
 ----------------
 
 - Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266))
 - Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322))
 - Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480))
 - Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362))
 - Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379))
 - Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392))
 - Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423))
 - Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429))
 - Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487))
 - Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482))
 - Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483))
 - Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEETQ1YthIGLQRddG54CTxDAAxPS/QFAl3uYYkACgkQCTxDAAxP
 S/QWMQ/8CKgO5cFl6kMxS6EjZJ6C8rA2QxP0iEcveqMj++yT3pQmdtQ5zzO100tS
 ZZYAdQbHkk/HWW8TIUtlxC+pcb9ZL0sZejY6dXzhD0dBVkFfhXQhJeFAkZnlf5lu
 CR8/kKTJ9IMgL6WtseOCoZQV/N9ifurWqqb8d0J5H5w1i//CP8opUQTn1a7lUzgm
 +1KleBv+A+gfltcm+NuC3+Q0GJNqfmtjcneOGwXXiuS3Ux8r2uLpbQsQfuNMMRkP
 ItmLJLy8hkP2yncrM9NE1Jkh0uOwcBJczbmT1OpXDIJiueFl7FxLKFXQM17QrpOT
 AqVDnVTlXfTDvWFeFstNz84uceXz98tLKZIl0X0inkN/wo3ZxMmzzxQw2Ob55ZPY
 pT6oC1gm0bGNQ/smEonyMM3z0ir7OfAce/tFQigfo59qeA08uzeckWkWpiXTbmy5
 xu2lomhpQABBNBhEnVpm4+dMXeXWzz1NtkeC6auBn+eX19JqlWZM9WcuNfv4rC+M
 eg5mwv2qERBA+qLsZQLy0dEMUy9fIrjd0GklMl6GUKYiSAsbDIptYMZl504D9Emc
 adTY63BT7TTiu64lr+CiQ9BbWQkGJxM1MQ8BoYyaKvgtahmOi+0Y5R0RGmrG6Hju
 UhgMdjmt5HnN0wiepBFCth9nJq+XDdyls8xAKGtSzGt129krBP4=
 =queo
 -----END PGP SIGNATURE-----

Merge tag 'v1.7.0rc1' into develop

Synapse 1.7.0rc1 (2019-12-09)
=============================

Features
--------

- Implement per-room message retention policies. ([\#5815](https://github.com/matrix-org/synapse/issues/5815), [\#6436](https://github.com/matrix-org/synapse/issues/6436))
- Add etag and count fields to key backup endpoints to help clients guess if there are new keys. ([\#5858](https://github.com/matrix-org/synapse/issues/5858))
- Add `/admin/v2/users` endpoint with pagination. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925))
- Require User-Interactive Authentication for `/account/3pid/add`, meaning the user's password will be required to add a third-party ID to their account. ([\#6119](https://github.com/matrix-org/synapse/issues/6119))
- Implement the `/_matrix/federation/unstable/net.atleastfornow/state/<context>` API as drafted in MSC2314. ([\#6176](https://github.com/matrix-org/synapse/issues/6176))
- Configure privacy-preserving settings by default for the room directory. ([\#6354](https://github.com/matrix-org/synapse/issues/6354))
- Add ephemeral messages support by partially implementing [MSC2228](https://github.com/matrix-org/matrix-doc/pull/2228). ([\#6409](https://github.com/matrix-org/synapse/issues/6409))
- Add support for [MSC 2367](https://github.com/matrix-org/matrix-doc/pull/2367), which allows specifying a reason on all membership events. ([\#6434](https://github.com/matrix-org/synapse/issues/6434))

Bugfixes
--------

- Transfer non-standard power levels on room upgrade. ([\#6237](https://github.com/matrix-org/synapse/issues/6237))
- Fix error from the Pillow library when uploading RGBA images. ([\#6241](https://github.com/matrix-org/synapse/issues/6241))
- Correctly apply the event filter to the `state`, `events_before` and `events_after` fields in the response to `/context` requests. ([\#6329](https://github.com/matrix-org/synapse/issues/6329))
- Fix caching devices for remote users when using workers, so that we don't attempt to refetch (and potentially fail) each time a user requests devices. ([\#6332](https://github.com/matrix-org/synapse/issues/6332))
- Prevent account data syncs getting lost across TCP replication. ([\#6333](https://github.com/matrix-org/synapse/issues/6333))
- Fix bug: TypeError in `register_user()` while using LDAP auth module. ([\#6406](https://github.com/matrix-org/synapse/issues/6406))
- Fix an intermittent exception when handling read-receipts. ([\#6408](https://github.com/matrix-org/synapse/issues/6408))
- Fix broken guest registration when there are existing blocks of numeric user IDs. ([\#6420](https://github.com/matrix-org/synapse/issues/6420))
- Fix startup error when http proxy is defined. ([\#6421](https://github.com/matrix-org/synapse/issues/6421))
- Fix error when using synapse_port_db on a vanilla synapse db. ([\#6449](https://github.com/matrix-org/synapse/issues/6449))
- Fix uploading multiple cross signing signatures for the same user. ([\#6451](https://github.com/matrix-org/synapse/issues/6451))
- Fix bug which lead to exceptions being thrown in a loop when a cross-signed device is deleted. ([\#6462](https://github.com/matrix-org/synapse/issues/6462))
- Fix `synapse_port_db` not exiting with a 0 code if something went wrong during the port process. ([\#6470](https://github.com/matrix-org/synapse/issues/6470))
- Improve sanity-checking when receiving events over federation. ([\#6472](https://github.com/matrix-org/synapse/issues/6472))
- Fix inaccurate per-block Prometheus metrics. ([\#6491](https://github.com/matrix-org/synapse/issues/6491))
- Fix small performance regression for sending invites. ([\#6493](https://github.com/matrix-org/synapse/issues/6493))
- Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression. ([\#6494](https://github.com/matrix-org/synapse/issues/6494))

Improved Documentation
----------------------

- Update documentation and variables in user contributed systemd reference file. ([\#6369](https://github.com/matrix-org/synapse/issues/6369), [\#6490](https://github.com/matrix-org/synapse/issues/6490))
- Fix link in the user directory documentation. ([\#6388](https://github.com/matrix-org/synapse/issues/6388))
- Add build instructions to the docker readme. ([\#6390](https://github.com/matrix-org/synapse/issues/6390))
- Switch Ubuntu package install recommendation to use python3 packages in INSTALL.md. ([\#6443](https://github.com/matrix-org/synapse/issues/6443))
- Write some docs for the quarantine_media api. ([\#6458](https://github.com/matrix-org/synapse/issues/6458))
- Convert CONTRIBUTING.rst to markdown (among other small fixes). ([\#6461](https://github.com/matrix-org/synapse/issues/6461))

Deprecations and Removals
-------------------------

- Remove admin/v1/users_paginate endpoint. Contributed by Awesome Technologies Innovationslabor GmbH. ([\#5925](https://github.com/matrix-org/synapse/issues/5925))
- Remove fallback for federation with old servers which lack the /federation/v1/state_ids API. ([\#6488](https://github.com/matrix-org/synapse/issues/6488))

Internal Changes
----------------

- Add benchmarks for structured logging and improve output performance. ([\#6266](https://github.com/matrix-org/synapse/issues/6266))
- Improve the performance of outputting structured logging. ([\#6322](https://github.com/matrix-org/synapse/issues/6322))
- Refactor some code in the event authentication path for clarity. ([\#6343](https://github.com/matrix-org/synapse/issues/6343), [\#6468](https://github.com/matrix-org/synapse/issues/6468), [\#6480](https://github.com/matrix-org/synapse/issues/6480))
- Clean up some unnecessary quotation marks around the codebase. ([\#6362](https://github.com/matrix-org/synapse/issues/6362))
- Complain on startup instead of 500'ing during runtime when `public_baseurl` isn't set when necessary. ([\#6379](https://github.com/matrix-org/synapse/issues/6379))
- Add a test scenario to make sure room history purges don't break `/messages` in the future. ([\#6392](https://github.com/matrix-org/synapse/issues/6392))
- Clarifications for the email configuration settings. ([\#6423](https://github.com/matrix-org/synapse/issues/6423))
- Add more tests to the blacklist when running in worker mode. ([\#6429](https://github.com/matrix-org/synapse/issues/6429))
- Refactor data store layer to support multiple databases in the future. ([\#6454](https://github.com/matrix-org/synapse/issues/6454), [\#6464](https://github.com/matrix-org/synapse/issues/6464), [\#6469](https://github.com/matrix-org/synapse/issues/6469), [\#6487](https://github.com/matrix-org/synapse/issues/6487))
- Port synapse.rest.client.v1 to async/await. ([\#6482](https://github.com/matrix-org/synapse/issues/6482))
- Port synapse.rest.client.v2_alpha to async/await. ([\#6483](https://github.com/matrix-org/synapse/issues/6483))
- Port SyncHandler to async/await. ([\#6484](https://github.com/matrix-org/synapse/issues/6484))
 2019-12-09 15:04:08 +00:00
Andrew Morgan
5e8abe9013
Better errors regarding changing avatar_url (#6497) 2019-12-09 14:54:33 +00:00
Neil Johnson
24da1ffcb6 1.7.0rc1 2019-12-09 14:46:20 +00:00
Neil Johnson
adfdd82b21
Back out perf regression from get_cross_signing_keys_from_cache. (#6494) 
Back out cross-signing code added in Synapse 1.5.0, which caused a performance regression.
 2019-12-09 13:59:27 +00:00
Erik Johnston
30e9adf32f
Merge pull request #6487 from matrix-org/erikj/pass_in_db 
Pass in Database object to data stores.
 2019-12-09 13:53:21 +00:00
Erik Johnston
e1544b0af8
Merge pull request #6493 from matrix-org/erikj/invite_state_config 
Pull out room_invite_state_types config option once.
 2019-12-09 12:23:21 +00:00
Erik Johnston
65b37f6729 Fix comment 
Co-Authored-By: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
 2019-12-09 11:56:43 +00:00
Richard van der Hoff
18660a34d8
Fix inaccurate per-block metrics (#6491) 
`Measure` incorrectly assumed that it was the only thing being done by the parent `LoggingContext`. For instance, during a "renew group attestations" operation, hundreds of `outbound_request` calls could take place in parallel, all using the same `LoggingContext`. This would mean that any resources used during *any* of those calls would be reported against *all* of them, producing wildly inaccurate results.

Instead, we now give each `Measure` block its own `LoggingContext` (using the parent `LoggingContext` mechanism to ensure that the log lines look correct and that the metrics are ultimately propogated to the top level for reporting against requests/backgrond tasks).
 2019-12-09 11:55:30 +00:00
Erik Johnston
8ad8bcbed0 Pull out room_invite_state_types config option once. 
Pulling things out of config is currently surprisingly expensive.
 2019-12-09 11:52:19 +00:00
Richard van der Hoff
e519489fc4
Remove fallback for missing /federation/v1/state_ids API (#6488) 
This API was added way back in 0.17.0; the code here is annoying to maintain
and entirely redundant.
 2019-12-09 11:37:26 +00:00
Erik Johnston
a9b393340f
Merge pull request #6484 from matrix-org/erikj/port_sync_handler 
Port SyncHandler to async/await
 2019-12-09 11:32:44 +00:00
Erik Johnston
71ee22c0ba Fix port db script 2019-12-06 16:41:48 +00:00
Erik Johnston
75f87450d8 Move start up DB checks to main data store. 2019-12-06 16:02:21 +00:00
Erik Johnston
d537be1ebd Pass Database into the data store 2019-12-06 15:49:44 +00:00
Erik Johnston
d64bb32a73 Move are_all_users_on_domain checks to main data store. 2019-12-06 13:43:40 +00:00
Erik Johnston
9a4fb457cf Change DataStores to accept 'database' param. 2019-12-06 13:30:06 +00:00
Erik Johnston
2ace775d88 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_database_class 2019-12-06 11:33:34 +00:00
Erik Johnston
e216ec381a Remove unused var 2019-12-06 11:16:37 +00:00
Erik Johnston
b3a4e35ca8 Fixup functions to consistently return deferreds 2019-12-06 10:40:05 +00:00
Manuel Stahl
649b6bc088 Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925) 2019-12-05 18:12:23 +00:00
Erik Johnston
8437e2383e Port SyncHandler to async/await 2019-12-05 17:58:25 +00:00
Erik Johnston
9c41ba4c5f Port rest.client.v2 2019-12-05 16:56:23 +00:00
Erik Johnston
1a0997bbd5 Port rest/v1 to async/await 2019-12-05 15:57:28 +00:00
Richard van der Hoff
63d6ad1064
Stronger typing in the federation handler (#6480) 
replace the event_info dict with an attrs thing
 2019-12-05 15:02:35 +00:00
Richard van der Hoff
e1f4c83f41
Sanity-check the rooms of auth events before pulling them in. (#6472) 2019-12-05 14:14:45 +00:00
Erik Johnston
4a33a6dd19 Move background update handling out of store 2019-12-05 11:11:26 +00:00
Erik Johnston
8863624f78 Comments 2019-12-05 10:46:37 +00:00
Erik Johnston
756d4942f5 Move DB pool and helper functions into dedicated Database class 2019-12-05 10:46:37 +00:00
Erik Johnston
ddbbfc9512
Merge pull request #6464 from matrix-org/erikj/make_public_sql_base 
Clean up SQLBaseStore private function usage
 2019-12-05 10:43:49 +00:00
Brendan Abolivier
fe799f353d
Merge pull request #6470 from matrix-org/babolivier/port_db_ci_failure 
Make synapse_port_db exit with a non-0 code if something failed
 2019-12-04 18:20:36 +00:00
Brendan Abolivier
f8421a1404
Fix background updates for synapse_port_db 2019-12-04 17:57:35 +00:00
Richard van der Hoff
e203874caa
get rid of (most of) have_events from _update_auth_events_and_context_for_auth (#6468) 
have_events was a map from event_id to rejection reason (or None) for events
which are in our local database. It was used as filter on the list of
event_ids being passed into get_events_as_list. However, since
get_events_as_list will ignore any event_ids that are unknown or rejected, we
can equivalently just leave it to get_events_as_list to do the filtering.

That means that we don't have to keep `have_events` up-to-date, and can use
`have_seen_events` instead of `get_seen_events_with_rejection` in the one place
we do need it.
 2019-12-04 17:27:32 +00:00
Erik Johnston
ee86abb2d6 Remove underscore from SQLBaseStore functions 2019-12-04 16:23:43 +00:00
Erik Johnston
c2f525a525 Don't call SQLBaseStore methods from outside stores 2019-12-04 16:23:43 +00:00
Erik Johnston
3eb15c01d9
Merge pull request #6454 from matrix-org/erikj/clean_base_Store 
Move things out of SQLBaseStore
 2019-12-04 16:23:19 +00:00
Erik Johnston
9186c105a0 Revert "Move get_user_count_txn out of base store" 
This reverts commit 00f0d67566.

Its going to get removed soon, so lets not make merge conflicts.
 2019-12-04 15:46:19 +00:00
Erik Johnston
a7f20500ff _CURRENT_STATE_CACHE_NAME is public 2019-12-04 15:45:42 +00:00
Erik Johnston
b9449012db
Merge pull request #6441 from syamgk/fix-parameter-mismatch 
Fix issue #6406 parameter mismatch
 2019-12-04 15:42:33 +00:00
Brendan Abolivier
c530f9af4d
Merge pull request #6329 from matrix-org/babolivier/context_filters 
Filter state, events_before and events_after in /context requests
 2019-12-04 15:24:16 +00:00
Erik Johnston
00f0d67566 Move get_user_count_txn out of base store 2019-12-04 15:21:14 +00:00
Erik Johnston
1056d6885a Move cache invalidation to main data store 2019-12-04 15:21:14 +00:00
Erik Johnston
6b2867096b Move event fetch vars to EventWorkStore 2019-12-04 15:21:14 +00:00