Merge branch 'master' into develop

CHANGES.md

@@ -1,3 +1,91 @@
+Synapse 1.61.0rc1 (2022-06-07)
+==============================
+
+This release removes support for the non-standard feature known both as 'groups' and as 'communities', which have been superseded by *Spaces*.
+
+Features
+--------
+
+- Add new `media_retention` options to the homeserver config for routinely cleaning up non-recently accessed media. ([\#12732](https://github.com/matrix-org/synapse/issues/12732), [\#12972](https://github.com/matrix-org/synapse/issues/12972), [\#12977](https://github.com/matrix-org/synapse/issues/12977))
+- Experimental support for [MSC3772](https://github.com/matrix-org/matrix-spec-proposals/pull/3772): Push rule for mutually related events. ([\#12740](https://github.com/matrix-org/synapse/issues/12740), [\#12859](https://github.com/matrix-org/synapse/issues/12859))
+- Update to the `check_event_for_spam` module callback: Deprecate the current callback signature, replace it with a new signature that is both less ambiguous (replacing booleans with explicit allow/block) and more powerful (ability to return explicit error codes). ([\#12808](https://github.com/matrix-org/synapse/issues/12808))
+- Add storage and module API methods to get monthly active users (and their corresponding appservices) within an optionally specified time range. ([\#12838](https://github.com/matrix-org/synapse/issues/12838), [\#12917](https://github.com/matrix-org/synapse/issues/12917))
+- Support the new error code `ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED` from [MSC3823](https://github.com/matrix-org/matrix-spec-proposals/pull/3823). ([\#12845](https://github.com/matrix-org/synapse/issues/12845), [\#12923](https://github.com/matrix-org/synapse/issues/12923))
+- Add a configurable background job to delete stale devices. ([\#12855](https://github.com/matrix-org/synapse/issues/12855))
+- Improve URL previews for pages with empty elements. ([\#12951](https://github.com/matrix-org/synapse/issues/12951))
+- Allow updating a user's password using the admin API without logging out their devices. Contributed by @jcgruenhage. ([\#12952](https://github.com/matrix-org/synapse/issues/12952))
+
+
+Bugfixes
+--------
+
+- Always send an `access_token` in `/thirdparty/` requests to appservices, as required by the [Application Service API specification](https://spec.matrix.org/v1.1/application-service-api/#third-party-networks). ([\#12746](https://github.com/matrix-org/synapse/issues/12746))
+- Implement [MSC3816](https://github.com/matrix-org/matrix-spec-proposals/pull/3816): sending the root event in a thread should count as having 'participated' in it. ([\#12766](https://github.com/matrix-org/synapse/issues/12766))
+- Delete events from the `federation_inbound_events_staging` table when a room is purged through the admin API. ([\#12784](https://github.com/matrix-org/synapse/issues/12784))
+- Fix a bug where we did not correctly handle invalid device list updates over federation. Contributed by Carl Bordum Hansen. ([\#12829](https://github.com/matrix-org/synapse/issues/12829))
+- Fix a bug which allowed multiple async operations to access database locks concurrently. Contributed by @sumnerevans @ Beeper. ([\#12832](https://github.com/matrix-org/synapse/issues/12832))
+- Fix an issue introduced in Synapse 0.34 where the `/notifications` endpoint would only return notifications if a user registered at least one pusher. Contributed by Famedly. ([\#12840](https://github.com/matrix-org/synapse/issues/12840))
+- Fix a bug where servers using a Postgres database would fail to backfill from an insertion event when MSC2716 is enabled (`experimental_features.msc2716_enabled`). ([\#12843](https://github.com/matrix-org/synapse/issues/12843))
+- Fix [MSC3787](https://github.com/matrix-org/matrix-spec-proposals/pull/3787) rooms being omitted from room directory, room summary and space hierarchy responses. ([\#12858](https://github.com/matrix-org/synapse/issues/12858))
+- Fix a bug introduced in Synapse 1.54.0 which could sometimes cause exceptions when handling federated traffic. ([\#12877](https://github.com/matrix-org/synapse/issues/12877))
+- Fix a bug introduced in Synapse 1.59.0 which caused room deletion to fail with a foreign key violation error. ([\#12889](https://github.com/matrix-org/synapse/issues/12889))
+- Fix a long-standing bug which caused the `/messages` endpoint to return an incorrect `end` attribute when there were no more events. Contributed by @Vetchu. ([\#12903](https://github.com/matrix-org/synapse/issues/12903))
+- Fix a bug introduced in Synapse 1.58.0 where `/sync` would fail if the most recent event in a room was a redaction of an event that has since been purged. ([\#12905](https://github.com/matrix-org/synapse/issues/12905))
+- Fix a potential memory leak when generating thumbnails. ([\#12932](https://github.com/matrix-org/synapse/issues/12932))
+- Fix a long-standing bug where a URL preview would break if the image failed to download. ([\#12950](https://github.com/matrix-org/synapse/issues/12950))
+
+
+Improved Documentation
+----------------------
+
+- Fix typographical errors in documentation. ([\#12863](https://github.com/matrix-org/synapse/issues/12863))
+- Fix documentation incorrectly stating the `sendToDevice` endpoint can be directed at generic workers. Contributed by Nick @ Beeper. ([\#12867](https://github.com/matrix-org/synapse/issues/12867))
+
+
+Deprecations and Removals
+-------------------------
+
+- Remove support for the non-standard groups/communities feature from Synapse. ([\#12553](https://github.com/matrix-org/synapse/issues/12553), [\#12558](https://github.com/matrix-org/synapse/issues/12558), [\#12563](https://github.com/matrix-org/synapse/issues/12563), [\#12895](https://github.com/matrix-org/synapse/issues/12895), [\#12897](https://github.com/matrix-org/synapse/issues/12897), [\#12899](https://github.com/matrix-org/synapse/issues/12899), [\#12900](https://github.com/matrix-org/synapse/issues/12900), [\#12936](https://github.com/matrix-org/synapse/issues/12936), [\#12966](https://github.com/matrix-org/synapse/issues/12966))
+- Remove contributed `kick_users.py` script. This is broken under Python 3, and is not added to the environment when `pip install`ing Synapse. ([\#12908](https://github.com/matrix-org/synapse/issues/12908))
+- Remove `contrib/jitsimeetbridge`. This was an unused experiment that hasn't been meaningfully changed since 2014. ([\#12909](https://github.com/matrix-org/synapse/issues/12909))
+- Remove unused `contrib/experiements/cursesio.py` script, which fails to run under Python 3. ([\#12910](https://github.com/matrix-org/synapse/issues/12910))
+- Remove unused `contrib/experiements/test_messaging.py` script. This fails to run on Python 3. ([\#12911](https://github.com/matrix-org/synapse/issues/12911))
+
+
+Internal Changes
+----------------
+
+- Test Synapse against Complement with workers. ([\#12810](https://github.com/matrix-org/synapse/issues/12810), [\#12933](https://github.com/matrix-org/synapse/issues/12933))
+- Reduce the amount of state we pull from the DB. ([\#12811](https://github.com/matrix-org/synapse/issues/12811), [\#12964](https://github.com/matrix-org/synapse/issues/12964))
+- Try other homeservers when re-syncing state for rooms with partial state. ([\#12812](https://github.com/matrix-org/synapse/issues/12812))
+- Resume state re-syncing for rooms with partial state after a Synapse restart. ([\#12813](https://github.com/matrix-org/synapse/issues/12813))
+- Remove Mutual Rooms' ([MSC2666](https://github.com/matrix-org/matrix-spec-proposals/pull/2666)) endpoint dependency on the User Directory. ([\#12836](https://github.com/matrix-org/synapse/issues/12836))
+- Experimental: expand `check_event_for_spam` with ability to return additional fields. This enables spam-checker implementations to experiment with mechanisms to give users more information about why they are blocked and whether any action is needed from them to be unblocked. ([\#12846](https://github.com/matrix-org/synapse/issues/12846))
+- Remove `dont_notify` from the `.m.rule.room.server_acl` rule. ([\#12849](https://github.com/matrix-org/synapse/issues/12849))
+- Remove the unstable `/hierarchy` endpoint from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946). ([\#12851](https://github.com/matrix-org/synapse/issues/12851))
+- Pull out less state when handling gaps in room DAG. ([\#12852](https://github.com/matrix-org/synapse/issues/12852), [\#12904](https://github.com/matrix-org/synapse/issues/12904))
+- Clean-up the push rules datastore. ([\#12856](https://github.com/matrix-org/synapse/issues/12856))
+- Correct a type annotation in the URL preview source code. ([\#12860](https://github.com/matrix-org/synapse/issues/12860))
+- Update `pyjwt` dependency to [2.4.0](https://github.com/jpadilla/pyjwt/releases/tag/2.4.0). ([\#12865](https://github.com/matrix-org/synapse/issues/12865))
+- Enable the `/account/whoami` endpoint on synapse worker processes. Contributed by Nick @ Beeper. ([\#12866](https://github.com/matrix-org/synapse/issues/12866))
+- Enable the `batch_send` endpoint on synapse worker processes. Contributed by Nick @ Beeper. ([\#12868](https://github.com/matrix-org/synapse/issues/12868))
+- Don't generate empty AS transactions when the AS is flagged as down. Contributed by Nick @ Beeper. ([\#12869](https://github.com/matrix-org/synapse/issues/12869))
+- Fix up the variable `state_store` naming. ([\#12871](https://github.com/matrix-org/synapse/issues/12871))
+- Faster room joins: when querying the current state of the room, wait for state to be populated. ([\#12872](https://github.com/matrix-org/synapse/issues/12872))
+- Avoid running queries which will never result in deletions. ([\#12879](https://github.com/matrix-org/synapse/issues/12879))
+- Use constants for EDU types. ([\#12884](https://github.com/matrix-org/synapse/issues/12884))
+- Reduce database load of `/sync` when presence is enabled. ([\#12885](https://github.com/matrix-org/synapse/issues/12885))
+- Refactor `have_seen_events` to reduce memory consumed when processing federation traffic. ([\#12886](https://github.com/matrix-org/synapse/issues/12886))
+- Refactor receipt linearization code. ([\#12888](https://github.com/matrix-org/synapse/issues/12888))
+- Add type annotations to `synapse.logging.opentracing`. ([\#12894](https://github.com/matrix-org/synapse/issues/12894))
+- Remove PyNaCl occurrences directly used in Synapse code. ([\#12902](https://github.com/matrix-org/synapse/issues/12902))
+- Bump types-jsonschema from 4.4.1 to 4.4.6. ([\#12912](https://github.com/matrix-org/synapse/issues/12912))
+- Rename storage classes. ([\#12913](https://github.com/matrix-org/synapse/issues/12913))
+- Preparation for database schema simplifications: stop reading from `event_edges.room_id`. ([\#12914](https://github.com/matrix-org/synapse/issues/12914))
+- Check if we are in a virtual environment before overriding the `PYTHONPATH` environment variable in the demo script. ([\#12916](https://github.com/matrix-org/synapse/issues/12916))
+- Improve the logging when signature checks on events fail. ([\#12925](https://github.com/matrix-org/synapse/issues/12925))
+
+
 Synapse 1.60.0 (2022-05-31)
 ===========================
 

changelog.d/12553.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12558.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12563.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12732.feature

@@ -1 +0,0 @@
-Add new `media_retention` options to the homeserver config for routinely cleaning up non-recently accessed media.

changelog.d/12740.feature

@@ -1 +0,0 @@
-Experimental support for [MSC3772](https://github.com/matrix-org/matrix-spec-proposals/pull/3772): Push rule for mutually related events.

changelog.d/12746.bugfix

@@ -1 +0,0 @@
-Always send an `access_token` in `/thirdparty/` requests to appservices, as required by the [Matrix specification](https://spec.matrix.org/v1.1/application-service-api/#third-party-networks).

changelog.d/12766.bugfix

@@ -1 +0,0 @@
-Implement [MSC3816](https://github.com/matrix-org/matrix-spec-proposals/pull/3816): sending the root event in a thread should count as "participated" in it.

changelog.d/12784.bugfix

@@ -1 +0,0 @@
-Delete events from the `federation_inbound_events_staging` table when a room is purged through the admin API.

changelog.d/12808.feature

@@ -1 +0,0 @@
-Update to `check_event_for_spam`. Deprecate the current callback signature, replace it with a new signature that is both less ambiguous (replacing booleans with explicit allow/block) and more powerful (ability to return explicit error codes).

changelog.d/12810.misc

@@ -1 +0,0 @@
-Test Synapse against Complement with workers.

changelog.d/12811.misc

@@ -1 +0,0 @@
-Reduce the amount of state we pull from the DB.

changelog.d/12812.misc

@@ -1 +0,0 @@
-Try other homeservers when re-syncing state for rooms with partial state.

changelog.d/12813.misc

@@ -1 +0,0 @@
-Resume state re-syncing for rooms with partial state after a Synapse restart.

changelog.d/12829.bugfix

@@ -1 +0,0 @@
-Fix a bug where we did not correctly handle invalid device list updates over federation. Contributed by Carl Bordum Hansen.

changelog.d/12832.bugfix

@@ -1 +0,0 @@
-Fixed a bug which allowed multiple async operations to access database locks concurrently. Contributed by @sumnerevans @ Beeper.

changelog.d/12836.misc

@@ -1 +0,0 @@
-Remove Mutual Rooms ([MSC2666](https://github.com/matrix-org/matrix-spec-proposals/pull/2666)) endpoint dependency on the User Directory.

changelog.d/12838.feature

@@ -1 +0,0 @@
-Add storage and module API methods to get monthly active users (and their corresponding appservices) within an optionally specified time range.

changelog.d/12840.bugfix

@@ -1 +0,0 @@
-Fix an issue introduced in Synapse 0.34 where the `/notifications` endpoint would only return notifications if a user registered at least one pusher. Contributed by Famedly.

changelog.d/12843.bugfix

@@ -1 +0,0 @@
-Fix bug where servers using a Postgres database would fail to backfill from an insertion event when MSC2716 is enabled (`experimental_features.msc2716_enabled`).

changelog.d/12845.feature

@@ -1 +0,0 @@
-Support the new error code "ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED" from [MSC3823](https://github.com/matrix-org/matrix-spec-proposals/pull/3823).

changelog.d/12846.misc

@@ -1 +0,0 @@
-Experimental: expand `check_event_for_spam` with ability to return additional fields. This enables spam-checker implementations to experiment with mechanisms to give users more information about why they are blocked and whether any action is needed from them to be unblocked.

changelog.d/12849.misc

@@ -1 +0,0 @@
-Remove `dont_notify` from the `.m.rule.room.server_acl` rule.

changelog.d/12851.misc

@@ -1 +0,0 @@
-Remove the unstable `/hierarchy` endpoint from [MSC2946](https://github.com/matrix-org/matrix-doc/pull/2946).

changelog.d/12852.misc

@@ -1 +0,0 @@
-Pull out less state when handling gaps in room DAG.

changelog.d/12855.feature

@@ -1 +0,0 @@
-Add a configurable background job to delete stale devices.

changelog.d/12856.misc

@@ -1 +0,0 @@
-Clean-up the push rules datastore.

changelog.d/12858.bugfix

@@ -1 +0,0 @@
-Fix [MSC3787](https://github.com/matrix-org/matrix-spec-proposals/pull/3787) rooms being omitted from room directory, room summary and space hierarchy responses.

changelog.d/12859.feature

@@ -1 +0,0 @@
-Experimental support for [MSC3772](https://github.com/matrix-org/matrix-spec-proposals/pull/3772): Push rule for mutually related events.

changelog.d/12860.misc

@@ -1 +0,0 @@
-Correct a type annotation in the URL preview source code.

changelog.d/12863.doc

@@ -1 +0,0 @@
-Fix typos in documentation.

changelog.d/12865.misc

@@ -1 +0,0 @@
-Update `pyjwt` dependency to [2.4.0](https://github.com/jpadilla/pyjwt/releases/tag/2.4.0).

changelog.d/12866.misc

@@ -1 +0,0 @@
-Enable the `/account/whoami` endpoint on synapse worker processes. Contributed by Nick @ Beeper.

changelog.d/12867.doc

@@ -1 +0,0 @@
-Fix documentation incorrectly stating the `sendToDevice` endpoint can be directed at generic workers. Contributed by Nick @ Beeper.

changelog.d/12868.misc

@@ -1 +0,0 @@
-Enable the `batch_send` endpoint on synapse worker processes. Contributed by Nick @ Beeper.

changelog.d/12869.misc

@@ -1 +0,0 @@
-Don't generate empty AS transactions when the AS is flagged as down. Contributed by Nick @ Beeper.

changelog.d/12871.misc

@@ -1 +0,0 @@
-Fix up the variable `state_store` naming.

changelog.d/12872.misc

@@ -1 +0,0 @@
-Faster room joins: when querying the current state of the room, wait for state to be populated.

changelog.d/12877.bugfix

@@ -1 +0,0 @@
-Fix a bug introduced in Synapse 1.54 which could sometimes cause exceptions when handling federated traffic.

changelog.d/12879.misc

@@ -1 +0,0 @@
-Avoid running queries which will never result in deletions.

changelog.d/12884.misc

@@ -1 +0,0 @@
-Use constants for EDU types.

changelog.d/12885.misc

@@ -1 +0,0 @@
-Reduce database load of `/sync` when presence is enabled.

changelog.d/12886.misc

@@ -1 +0,0 @@
-Refactor `have_seen_events` to reduce memory consumed when processing federation traffic.

changelog.d/12888.misc

@@ -1 +0,0 @@
-Refactor receipt linearization code.

changelog.d/12889.bugfix

@@ -1 +0,0 @@
-Fix a bug introduced in Synapse 1.59.0 which caused room deletion to fail with a foreign key violation.

changelog.d/12894.misc

@@ -1 +0,0 @@
-Add type annotations to `synapse.logging.opentracing`.

changelog.d/12895.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12897.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12899.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12900.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12902.misc

@@ -1 +0,0 @@
-Remove PyNaCl occurrences directly used in Synapse code.

changelog.d/12903.bugfix

@@ -1 +0,0 @@
-Fix a long-standing bug which caused the `/messages` endpoint to return an incorrect `end` attribute when there were no more events. Contributed by @Vetchu.

changelog.d/12904.misc

@@ -1 +0,0 @@
-Pull out less state when handling gaps in room DAG.

changelog.d/12905.bugfix

@@ -1 +0,0 @@
-Fix a bug introduced in Synapse 1.58.0 where `/sync` would fail if the most recent event in a room was a redaction of an event that has since been purged.

changelog.d/12908.removal

@@ -1 +0,0 @@
-Remove contributed `kick_users.py` script. This is broken under Python 3, and is not added to the environment when `pip install`ing Synapse.

changelog.d/12909.removal

@@ -1 +0,0 @@
-Remove `contrib/jitsimeetbridge`. This was an unused experiment that hasn't been meaningfully changed since 2014.

changelog.d/12910.removal

@@ -1 +0,0 @@
-Remove unused `contrib/experiements/cursesio.py` script, which fails to run under Python 3.

changelog.d/12911.removal

@@ -1 +0,0 @@
-Remove unused `contrib/experiements/test_messaging.py` script. This fails to run on Python 3.

changelog.d/12912.misc

@@ -1 +0,0 @@
-Bump types-jsonschema from 4.4.1 to 4.4.6.

changelog.d/12913.misc

@@ -1 +0,0 @@
-Rename storage classes.

changelog.d/12914.misc

@@ -1 +0,0 @@
-Preparation for database schema simplifications: stop reading from `event_edges.room_id`.

changelog.d/12916.misc

@@ -1 +0,0 @@
-Check if we are in a virtual environment before overriding the `PYTHONPATH` environment variable in the demo script.

changelog.d/12917.feature

@@ -1 +0,0 @@
-Add storage and module API methods to get monthly active users (and their corresponding appservices) within an optionally specified time range.

changelog.d/12923.feature

@@ -1 +0,0 @@
-Support the new error code "ORG.MATRIX.MSC3823.USER_ACCOUNT_SUSPENDED" from [MSC3823](https://github.com/matrix-org/matrix-spec-proposals/pull/3823).

changelog.d/12925.misc

@@ -1 +0,0 @@
-Improve the logging when signature checks on events fail.

changelog.d/12932.bugfix

@@ -1 +0,0 @@
-Fix potential memory leak when generating thumbnails.

changelog.d/12933.misc

@@ -1 +0,0 @@
-Test Synapse against Complement with workers.

changelog.d/12936.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12950.bugfix

@@ -1 +0,0 @@
-Fix a long-standing bug where a URL preview would break if the image failed to download.

changelog.d/12951.feature

@@ -1 +0,0 @@
-Improve URL previews for pages with empty elements.

changelog.d/12952.feature

@@ -1 +0,0 @@
-Allow updating a user's password using the admin API without logging out their devices. Contributed by @jcgruenhage.

changelog.d/12964.misc

@@ -1 +0,0 @@
-Reduce the amount of state we pull from the DB.

changelog.d/12966.removal

@@ -1 +0,0 @@
-Remove support for the non-standard groups/communities feature from Synapse.

changelog.d/12972.feature

@@ -1 +0,0 @@
-Add new `media_retention` options to the homeserver config for routinely cleaning up non-recently accessed media.

changelog.d/12977.feature

@@ -1 +0,0 @@
-Add new `media_retention` options to the homeserver config for routinely cleaning up non-recently accessed media.

debian/changelog

@@ -1,9 +1,9 @@
-matrix-synapse-py3 (1.61.0~rc1+nmu1) UNRELEASED; urgency=medium
+matrix-synapse-py3 (1.61.0~rc1) stable; urgency=medium
 
-  * Non-maintainer upload.
   * Remove unused `jitsimeetbridge` experiment from `contrib` directory.
+  * New Synapse release 1.61.0rc1.
 
- -- Synapse Packaging team <packages@matrix.org>  Sun, 29 May 2022 14:44:45 +0100
+ -- Synapse Packaging team <packages@matrix.org>  Tue, 07 Jun 2022 12:42:31 +0100
 
 matrix-synapse-py3 (1.60.0) stable; urgency=medium
 

pyproject.toml

@@ -54,7 +54,7 @@ skip_gitignore = true
 
 [tool.poetry]
 name = "matrix-synapse"
-version = "1.60.0"
+version = "1.61.0rc1"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "Apache-2.0"