diff --git a/CHANGES.md b/CHANGES.md index 82baaa2d1..066f798a9 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,3 +1,72 @@ +Synapse 1.39.0rc1 (2021-07-20) +============================== + +The Third-Party Event Rules module interface has been deprecated in favour of the generic module interface introduced in Synapse v1.37.0. Support for the old interface is planned to be removed in September 2021. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. + +Features +-------- + +- Add the ability to override the account validity feature with a module. ([\#9884](https://github.com/matrix-org/synapse/issues/9884)) +- The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. ([\#10298](https://github.com/matrix-org/synapse/issues/10298), [\#10305](https://github.com/matrix-org/synapse/issues/10305)) +- Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. ([\#10332](https://github.com/matrix-org/synapse/issues/10332), [\#10427](https://github.com/matrix-org/synapse/issues/10427)) +- Allow providing credentials to `http_proxy`. ([\#10360](https://github.com/matrix-org/synapse/issues/10360)) + + +Bugfixes +-------- + +- Fix error while dropping locks on shutdown. Introduced in v1.38.0. ([\#10433](https://github.com/matrix-org/synapse/issues/10433)) +- Add base starting insertion event when no chunk ID is specified in the historical batch send API. ([\#10250](https://github.com/matrix-org/synapse/issues/10250)) +- Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. ([\#10276](https://github.com/matrix-org/synapse/issues/10276)) +- Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. ([\#10317](https://github.com/matrix-org/synapse/issues/10317)) +- Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). ([\#10343](https://github.com/matrix-org/synapse/issues/10343)) +- Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. ([\#10344](https://github.com/matrix-org/synapse/issues/10344)) +- Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. ([\#10355](https://github.com/matrix-org/synapse/issues/10355)) +- Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. ([\#10359](https://github.com/matrix-org/synapse/issues/10359)) +- Fix `make_room_admin` failing for users that have left a private room. ([\#10367](https://github.com/matrix-org/synapse/issues/10367)) +- Fix a number of logged errors caused by remote servers being down. ([\#10400](https://github.com/matrix-org/synapse/issues/10400), [\#10414](https://github.com/matrix-org/synapse/issues/10414)) +- Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. ([\#10404](https://github.com/matrix-org/synapse/issues/10404)) + + +Improved Documentation +---------------------- + +- Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. ([\#9971](https://github.com/matrix-org/synapse/issues/9971)) +- Simplify structure of room admin API. ([\#10313](https://github.com/matrix-org/synapse/issues/10313)) +- Refresh the logcontext dev documentation. ([\#10353](https://github.com/matrix-org/synapse/issues/10353)), ([\#10337](https://github.com/matrix-org/synapse/issues/10337)) +- Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. ([\#10368](https://github.com/matrix-org/synapse/issues/10368)) +- Fix and clarify some links in `docs` and `contrib`. ([\#10370](https://github.com/matrix-org/synapse/issues/10370)), ([\#10322](https://github.com/matrix-org/synapse/issues/10322)), ([\#10399](https://github.com/matrix-org/synapse/issues/10399)) +- Make deprecation notice of the spam checker doc more obvious. ([\#10395](https://github.com/matrix-org/synapse/issues/10395)) +- Add instructions on installing Debian packages for release candidates. ([\#10396](https://github.com/matrix-org/synapse/issues/10396)) + + +Deprecations and Removals +------------------------- + +- Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. ([\#9721](https://github.com/matrix-org/synapse/issues/9721)) +- The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. ([\#10386](https://github.com/matrix-org/synapse/issues/10386)) + + +Internal Changes +---------------- + +- Convert `room_depth.min_depth` column to a `BIGINT`. ([\#10289](https://github.com/matrix-org/synapse/issues/10289)) +- Add tests to characterise the current behaviour of R30 phone-home metrics. ([\#10315](https://github.com/matrix-org/synapse/issues/10315)) +- Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. ([\#10316](https://github.com/matrix-org/synapse/issues/10316)) +- Minor change to the code that populates `user_daily_visits`. ([\#10324](https://github.com/matrix-org/synapse/issues/10324)) +- Re-enable Sytests that were disabled for the 1.37.1 release. ([\#10345](https://github.com/matrix-org/synapse/issues/10345), [\#10357](https://github.com/matrix-org/synapse/issues/10357)) +- Run `pyupgrade` on the codebase. ([\#10347](https://github.com/matrix-org/synapse/issues/10347), [\#10348](https://github.com/matrix-org/synapse/issues/10348)) +- Switch `application_services_txns.txn_id` database column to `BIGINT`. ([\#10349](https://github.com/matrix-org/synapse/issues/10349)) +- Convert internal type variable syntax to reflect wider ecosystem use. ([\#10350](https://github.com/matrix-org/synapse/issues/10350), [\#10380](https://github.com/matrix-org/synapse/issues/10380), [\#10381](https://github.com/matrix-org/synapse/issues/10381), [\#10382](https://github.com/matrix-org/synapse/issues/10382), [\#10418](https://github.com/matrix-org/synapse/issues/10418)) +- Make the Github Actions workflow configuration more efficient. ([\#10383](https://github.com/matrix-org/synapse/issues/10383)) +- Add type hints to `get_{domain,localpart}_from_id`. ([\#10385](https://github.com/matrix-org/synapse/issues/10385)) +- When building Debian packages for prerelease versions, set the Section accordingly. ([\#10391](https://github.com/matrix-org/synapse/issues/10391)) +- Add type hints and comments to event auth code. ([\#10393](https://github.com/matrix-org/synapse/issues/10393)) +- Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. ([\#10398](https://github.com/matrix-org/synapse/issues/10398)) +- Remove unused `events_by_room` code (tech debt). ([\#10421](https://github.com/matrix-org/synapse/issues/10421)) +- Add a github actions job which records success of other jobs. ([\#10430](https://github.com/matrix-org/synapse/issues/10430)) + + Synapse 1.38.0 (2021-07-13) =========================== diff --git a/changelog.d/10250.bugfix b/changelog.d/10250.bugfix deleted file mode 100644 index a8107dafb..000000000 --- a/changelog.d/10250.bugfix +++ /dev/null @@ -1 +0,0 @@ -Add base starting insertion event when no chunk ID is specified in the historical batch send API. diff --git a/changelog.d/10276.bugfix b/changelog.d/10276.bugfix deleted file mode 100644 index 42adc57ad..000000000 --- a/changelog.d/10276.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix historical batch send endpoint (MSC2716) rejecting batches with messages from multiple senders. diff --git a/changelog.d/10289.misc b/changelog.d/10289.misc deleted file mode 100644 index 2df30e7a7..000000000 --- a/changelog.d/10289.misc +++ /dev/null @@ -1 +0,0 @@ -Convert `room_depth.min_depth` column to a `BIGINT`. diff --git a/changelog.d/10298.feature b/changelog.d/10298.feature deleted file mode 100644 index 7059db507..000000000 --- a/changelog.d/10298.feature +++ /dev/null @@ -1 +0,0 @@ -The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. diff --git a/changelog.d/10305.feature b/changelog.d/10305.feature deleted file mode 100644 index 7059db507..000000000 --- a/changelog.d/10305.feature +++ /dev/null @@ -1 +0,0 @@ -The spaces summary API now returns any joinable rooms, not only rooms which are world-readable. diff --git a/changelog.d/10313.doc b/changelog.d/10313.doc deleted file mode 100644 index 44086e3d9..000000000 --- a/changelog.d/10313.doc +++ /dev/null @@ -1 +0,0 @@ -Simplify structure of room admin API. \ No newline at end of file diff --git a/changelog.d/10315.misc b/changelog.d/10315.misc deleted file mode 100644 index 2c78644e2..000000000 --- a/changelog.d/10315.misc +++ /dev/null @@ -1 +0,0 @@ -Add tests to characterise the current behaviour of R30 phone-home metrics. diff --git a/changelog.d/10316.misc b/changelog.d/10316.misc deleted file mode 100644 index 1fd0810fd..000000000 --- a/changelog.d/10316.misc +++ /dev/null @@ -1 +0,0 @@ -Rebuild event context and auth when processing specific results from `ThirdPartyEventRules` modules. diff --git a/changelog.d/10317.bugfix b/changelog.d/10317.bugfix deleted file mode 100644 index 826c269ef..000000000 --- a/changelog.d/10317.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix purging rooms that other homeservers are still sending events for. Contributed by @ilmari. diff --git a/changelog.d/10322.doc b/changelog.d/10322.doc deleted file mode 100644 index db604cf2a..000000000 --- a/changelog.d/10322.doc +++ /dev/null @@ -1 +0,0 @@ -Fix a broken link in the admin api docs. diff --git a/changelog.d/10324.misc b/changelog.d/10324.misc deleted file mode 100644 index 3c3ee6d6f..000000000 --- a/changelog.d/10324.misc +++ /dev/null @@ -1 +0,0 @@ -Minor change to the code that populates `user_daily_visits`. diff --git a/changelog.d/10332.feature b/changelog.d/10332.feature deleted file mode 100644 index 091947ff2..000000000 --- a/changelog.d/10332.feature +++ /dev/null @@ -1 +0,0 @@ -Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. diff --git a/changelog.d/10337.doc b/changelog.d/10337.doc deleted file mode 100644 index f305bdb3b..000000000 --- a/changelog.d/10337.doc +++ /dev/null @@ -1 +0,0 @@ -Fix formatting in the logcontext documentation. diff --git a/changelog.d/10343.bugfix b/changelog.d/10343.bugfix deleted file mode 100644 index 53ccf79a8..000000000 --- a/changelog.d/10343.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix errors during backfill caused by previously purged redaction events. Contributed by Andreas Rammhold (@andir). diff --git a/changelog.d/10344.bugfix b/changelog.d/10344.bugfix deleted file mode 100644 index ab6eb4999..000000000 --- a/changelog.d/10344.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix the user directory becoming broken (and noisy errors being logged) when knocking and room statistics are in use. diff --git a/changelog.d/10345.misc b/changelog.d/10345.misc deleted file mode 100644 index 7424486e8..000000000 --- a/changelog.d/10345.misc +++ /dev/null @@ -1 +0,0 @@ -Re-enable Sytests that were disabled for the 1.37.1 release. diff --git a/changelog.d/10347.misc b/changelog.d/10347.misc deleted file mode 100644 index b2275a135..000000000 --- a/changelog.d/10347.misc +++ /dev/null @@ -1 +0,0 @@ -Run `pyupgrade` on the codebase. \ No newline at end of file diff --git a/changelog.d/10348.misc b/changelog.d/10348.misc deleted file mode 100644 index b2275a135..000000000 --- a/changelog.d/10348.misc +++ /dev/null @@ -1 +0,0 @@ -Run `pyupgrade` on the codebase. \ No newline at end of file diff --git a/changelog.d/10349.misc b/changelog.d/10349.misc deleted file mode 100644 index 5b014e741..000000000 --- a/changelog.d/10349.misc +++ /dev/null @@ -1 +0,0 @@ -Switch `application_services_txns.txn_id` database column to `BIGINT`. diff --git a/changelog.d/10350.misc b/changelog.d/10350.misc deleted file mode 100644 index eed2d8552..000000000 --- a/changelog.d/10350.misc +++ /dev/null @@ -1 +0,0 @@ -Convert internal type variable syntax to reflect wider ecosystem use. \ No newline at end of file diff --git a/changelog.d/10353.doc b/changelog.d/10353.doc deleted file mode 100644 index 274ac8354..000000000 --- a/changelog.d/10353.doc +++ /dev/null @@ -1 +0,0 @@ -Refresh the logcontext dev documentation. diff --git a/changelog.d/10355.bugfix b/changelog.d/10355.bugfix deleted file mode 100644 index 92df61201..000000000 --- a/changelog.d/10355.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix newly added `synapse_federation_server_oldest_inbound_pdu_in_staging` prometheus metric to measure age rather than timestamp. diff --git a/changelog.d/10357.misc b/changelog.d/10357.misc deleted file mode 100644 index 7424486e8..000000000 --- a/changelog.d/10357.misc +++ /dev/null @@ -1 +0,0 @@ -Re-enable Sytests that were disabled for the 1.37.1 release. diff --git a/changelog.d/10359.bugfix b/changelog.d/10359.bugfix deleted file mode 100644 index d318f8fa0..000000000 --- a/changelog.d/10359.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix PostgreSQL sometimes using table scans for queries against `state_groups_state` table, taking a long time and a large amount of IO. diff --git a/changelog.d/10360.feature b/changelog.d/10360.feature deleted file mode 100644 index 904221cb6..000000000 --- a/changelog.d/10360.feature +++ /dev/null @@ -1 +0,0 @@ -Allow providing credentials to `http_proxy`. \ No newline at end of file diff --git a/changelog.d/10367.bugfix b/changelog.d/10367.bugfix deleted file mode 100644 index b44555608..000000000 --- a/changelog.d/10367.bugfix +++ /dev/null @@ -1 +0,0 @@ -Bugfix `make_room_admin` fails for users that have left a private room. \ No newline at end of file diff --git a/changelog.d/10368.doc b/changelog.d/10368.doc deleted file mode 100644 index 10297aa42..000000000 --- a/changelog.d/10368.doc +++ /dev/null @@ -1 +0,0 @@ -Add delegation example for caddy in the reverse proxy documentation. Contributed by @moritzdietz. diff --git a/changelog.d/10370.doc b/changelog.d/10370.doc deleted file mode 100644 index 8c59d98ee..000000000 --- a/changelog.d/10370.doc +++ /dev/null @@ -1 +0,0 @@ -Fix some links in `docs` and `contrib`. \ No newline at end of file diff --git a/changelog.d/10380.misc b/changelog.d/10380.misc deleted file mode 100644 index eed2d8552..000000000 --- a/changelog.d/10380.misc +++ /dev/null @@ -1 +0,0 @@ -Convert internal type variable syntax to reflect wider ecosystem use. \ No newline at end of file diff --git a/changelog.d/10381.misc b/changelog.d/10381.misc deleted file mode 100644 index eed2d8552..000000000 --- a/changelog.d/10381.misc +++ /dev/null @@ -1 +0,0 @@ -Convert internal type variable syntax to reflect wider ecosystem use. \ No newline at end of file diff --git a/changelog.d/10382.misc b/changelog.d/10382.misc deleted file mode 100644 index eed2d8552..000000000 --- a/changelog.d/10382.misc +++ /dev/null @@ -1 +0,0 @@ -Convert internal type variable syntax to reflect wider ecosystem use. \ No newline at end of file diff --git a/changelog.d/10383.misc b/changelog.d/10383.misc deleted file mode 100644 index 952c1e77a..000000000 --- a/changelog.d/10383.misc +++ /dev/null @@ -1 +0,0 @@ -Make the Github Actions workflow configuration more efficient. diff --git a/changelog.d/10385.misc b/changelog.d/10385.misc deleted file mode 100644 index e515ac09f..000000000 --- a/changelog.d/10385.misc +++ /dev/null @@ -1 +0,0 @@ -Add type hints to `get_{domain,localpart}_from_id`. diff --git a/changelog.d/10386.removal b/changelog.d/10386.removal deleted file mode 100644 index 800a6143d..000000000 --- a/changelog.d/10386.removal +++ /dev/null @@ -1 +0,0 @@ -The third-party event rules module interface is deprecated in favour of the generic module interface introduced in Synapse v1.37.0. See the [upgrade notes](https://matrix-org.github.io/synapse/latest/upgrade.html#upgrading-to-v1390) for more information. diff --git a/changelog.d/10391.misc b/changelog.d/10391.misc deleted file mode 100644 index 3f191b520..000000000 --- a/changelog.d/10391.misc +++ /dev/null @@ -1 +0,0 @@ -When building Debian packages for prerelease versions, set the Section accordingly. diff --git a/changelog.d/10393.misc b/changelog.d/10393.misc deleted file mode 100644 index e80f16d60..000000000 --- a/changelog.d/10393.misc +++ /dev/null @@ -1 +0,0 @@ -Add type hints and comments to event auth code. diff --git a/changelog.d/10395.doc b/changelog.d/10395.doc deleted file mode 100644 index 4bdaea76c..000000000 --- a/changelog.d/10395.doc +++ /dev/null @@ -1 +0,0 @@ -Make deprecation notice of the spam checker doc more obvious. diff --git a/changelog.d/10396.doc b/changelog.d/10396.doc deleted file mode 100644 index b521ad9cb..000000000 --- a/changelog.d/10396.doc +++ /dev/null @@ -1 +0,0 @@ -Add instructructions on installing Debian packages for release candidates. diff --git a/changelog.d/10398.misc b/changelog.d/10398.misc deleted file mode 100644 index 326e54655..000000000 --- a/changelog.d/10398.misc +++ /dev/null @@ -1 +0,0 @@ -Stagger sending of presence update to remote servers, reducing CPU spikes caused by starting many connections to remote servers at once. diff --git a/changelog.d/10399.doc b/changelog.d/10399.doc deleted file mode 100644 index b596ac562..000000000 --- a/changelog.d/10399.doc +++ /dev/null @@ -1 +0,0 @@ -Rewrite the text of links to be clearer in the documentation. diff --git a/changelog.d/10400.bugfix b/changelog.d/10400.bugfix deleted file mode 100644 index bfebed8d2..000000000 --- a/changelog.d/10400.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a number of logged errors caused by remote servers being down. diff --git a/changelog.d/10404.bugfix b/changelog.d/10404.bugfix deleted file mode 100644 index 2e095b640..000000000 --- a/changelog.d/10404.bugfix +++ /dev/null @@ -1 +0,0 @@ -Responses from `/make_{join,leave,knock}` no longer include signatures, which will turn out to be invalid after events are returned to `/send_{join,leave,knock}`. diff --git a/changelog.d/10414.bugfix b/changelog.d/10414.bugfix deleted file mode 100644 index bfebed8d2..000000000 --- a/changelog.d/10414.bugfix +++ /dev/null @@ -1 +0,0 @@ -Fix a number of logged errors caused by remote servers being down. diff --git a/changelog.d/10418.misc b/changelog.d/10418.misc deleted file mode 100644 index eed2d8552..000000000 --- a/changelog.d/10418.misc +++ /dev/null @@ -1 +0,0 @@ -Convert internal type variable syntax to reflect wider ecosystem use. \ No newline at end of file diff --git a/changelog.d/10421.misc b/changelog.d/10421.misc deleted file mode 100644 index 385cbe07a..000000000 --- a/changelog.d/10421.misc +++ /dev/null @@ -1 +0,0 @@ -Remove unused `events_by_room` code (tech debt). diff --git a/changelog.d/10427.feature b/changelog.d/10427.feature deleted file mode 100644 index 091947ff2..000000000 --- a/changelog.d/10427.feature +++ /dev/null @@ -1 +0,0 @@ -Add a new version of the R30 phone-home metric, which removes a false impression of retention given by the old R30 metric. diff --git a/changelog.d/10430.misc b/changelog.d/10430.misc deleted file mode 100644 index a017cf4ac..000000000 --- a/changelog.d/10430.misc +++ /dev/null @@ -1 +0,0 @@ -Add a github actions job recording success of other jobs. diff --git a/changelog.d/9721.removal b/changelog.d/9721.removal deleted file mode 100644 index da2ba48c8..000000000 --- a/changelog.d/9721.removal +++ /dev/null @@ -1 +0,0 @@ -Remove functionality associated with the unused `room_stats_historical` and `user_stats_historical` tables. Contributed by @xmunoz. diff --git a/changelog.d/9884.feature b/changelog.d/9884.feature deleted file mode 100644 index 525fd2f93..000000000 --- a/changelog.d/9884.feature +++ /dev/null @@ -1 +0,0 @@ -Add a module type for the account validity feature. diff --git a/changelog.d/9971.doc b/changelog.d/9971.doc deleted file mode 100644 index ada68f70c..000000000 --- a/changelog.d/9971.doc +++ /dev/null @@ -1 +0,0 @@ -Updated installation dependencies for newer macOS versions and ARM Macs. Contributed by Luke Walsh. diff --git a/debian/changelog b/debian/changelog index 43d26fc13..4d214c23b 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,3 +1,9 @@ +matrix-synapse-py3 (1.39.0~rc1) stable; urgency=medium + + * New synapse release 1.39.0rc1. + + -- Synapse Packaging team Tue, 20 Jul 2021 14:28:34 +0100 + matrix-synapse-py3 (1.38.0) stable; urgency=medium * New synapse release 1.38.0. diff --git a/synapse/__init__.py b/synapse/__init__.py index 5ecce24ee..46902adab 100644 --- a/synapse/__init__.py +++ b/synapse/__init__.py @@ -47,7 +47,7 @@ try: except ImportError: pass -__version__ = "1.38.0" +__version__ = "1.39.0rc1" if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)): # We import here so that we don't have to install a bunch of deps when diff --git a/synapse/storage/databases/main/lock.py b/synapse/storage/databases/main/lock.py index 774861074..3d1dff660 100644 --- a/synapse/storage/databases/main/lock.py +++ b/synapse/storage/databases/main/lock.py @@ -78,7 +78,11 @@ class LockStore(SQLBaseStore): """Called when the server is shutting down""" logger.info("Dropping held locks due to shutdown") - for (lock_name, lock_key), token in self._live_tokens.items(): + # We need to take a copy of the tokens dict as dropping the locks will + # cause the dictionary to change. + tokens = dict(self._live_tokens) + + for (lock_name, lock_key), token in tokens.items(): await self._drop_lock(lock_name, lock_key, token) logger.info("Dropped locks due to shutdown") diff --git a/tests/storage/databases/main/test_lock.py b/tests/storage/databases/main/test_lock.py index 9ca70e736..d326a1d6a 100644 --- a/tests/storage/databases/main/test_lock.py +++ b/tests/storage/databases/main/test_lock.py @@ -98,3 +98,16 @@ class LockTestCase(unittest.HomeserverTestCase): lock2 = self.get_success(self.store.try_acquire_lock("name", "key")) self.assertIsNotNone(lock2) + + def test_shutdown(self): + """Test that shutting down Synapse releases the locks""" + # Acquire two locks + lock = self.get_success(self.store.try_acquire_lock("name", "key1")) + self.assertIsNotNone(lock) + lock2 = self.get_success(self.store.try_acquire_lock("name", "key2")) + self.assertIsNotNone(lock2) + + # Now call the shutdown code + self.get_success(self.store._on_shutdown()) + + self.assertEqual(self.store._live_tokens, {})