David Robertson
aec69d2481
Another batch of type annotations ( #12726 )
2022-05-13 12:35:31 +01:00
Jess Porter
39bed28b28
SpamChecker metrics ( #12513 )
...
* add Measure blocks all over SpamChecker
Signed-off-by: jesopo <github@lolnerd.net>
* fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck
* better changelog entry
2022-05-13 12:17:38 +01:00
Niklas
c9fc2c0d22
Update issuer URL in example OIDC Keycloak config ( #12727 )
...
* Update openid.md
Newer versions of keycloak returning a 404 when using the `/auth` prefix.
Related: https://github.com/matrix-org/synapse/issues/12714
2022-05-13 10:15:51 +00:00
Andrew Morgan
57f6c496d0
URL preview cache expiry logs: INFO -> DEBUG, text clarifications ( #12720 )
2022-05-12 18:16:32 +01:00
David Robertson
17e1eb7749
Reduce the number of "untyped defs" ( #12716 )
2022-05-12 14:33:50 +00:00
Andy Balaam
de1e599b9d
add default_power_level_content_override config option. ( #12618 )
...
Co-authored-by: Matthew Hodgson <matthew@matrix.org>
2022-05-12 10:41:35 +00:00
Andrew Morgan
409573f6d0
Fix reference to the wrong symbol in the media admin api docs ( #12715 )
2022-05-12 09:29:37 +01:00
Sean Quah
bf7ce92bf7
Enable cancellation of GET /members and GET /state requests ( #12708 )
...
Enable cancellation of `GET /rooms/$room_id/members`,
`GET /rooms/$room_id/state` and
`GET /rooms/$room_id/state/$state_key/*` requests.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 17:22:34 +01:00
David Robertson
db10f2c037
No longer permit empty body when sending receipts ( #12709 )
2022-05-11 15:34:17 +00:00
Sean Quah
6ee61b9052
Complain if a federation endpoint has the @cancellable flag ( #12705 )
...
`BaseFederationServlet` wraps its endpoints in a bunch of async code
that has not been vetted for compatibility with cancellation.
Fail CI if a `@cancellable` flag is applied to a federation endpoint.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 14:52:26 +01:00
David Robertson
d38d242411
Reload cache factors from disk on SIGHUP ( #12673 )
2022-05-11 13:43:22 +00:00
Sean Quah
a559c8b0d9
Respect the @cancellable flag for ReplicationEndpoints ( #12700 )
...
While `ReplicationEndpoint`s register themselves via `JsonResource`,
they pass a method that calls the handler, instead of the handler itself,
to `register_paths`. As a result, `JsonResource` will not correctly pick
up the `@cancellable` flag and we have to apply it ourselves.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 12:25:39 +01:00
Sean Quah
9d8e380d2e
Respect the @cancellable flag for RestServlets and BaseFederationServlets ( #12699 )
...
Both `RestServlet`s and `BaseFederationServlet`s register their handlers
with `HttpServer.register_paths` / `JsonResource.register_paths`. Update
`JsonResource` to respect the `@cancellable` flag on handlers registered
in this way.
Although `ReplicationEndpoint` also registers itself using
`register_paths`, it does not pass the handler method that would have the
`@cancellable` flag directly, and so needs separate handling.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 12:25:13 +01:00
Sean Quah
dffecade7d
Respect the @cancellable flag for DirectServe{Html,Json}Resources ( #12698 )
...
`DirectServeHtmlResource` and `DirectServeJsonResource` both inherit
from `_AsyncResource`. These classes expect to be subclassed with
`_async_render_*` methods.
This commit has no effect on `JsonResource`, despite inheriting from
`_AsyncResource`. `JsonResource` has its own `_async_render` override
which will need to be updated separately.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-11 12:24:48 +01:00
Patrick Cloke
a4c75918b3
Remove unneeded ActionGenerator class. ( #12691 )
...
It simply passes through to `BulkPushRuleEvaluator`, which can be
called directly instead.
2022-05-11 07:15:21 -04:00
Eric Eastwood
84facf769e
Fix /messages throwing a 500 when querying for non-existent room ( #12683 )
...
Fix https://github.com/matrix-org/synapse/issues/12678
Complement test added: https://github.com/matrix-org/complement/pull/369
**Before:** 500 internal server error
**After:** According to the [spec](https://spec.matrix.org/latest/client-server-api/#get_matrixclientv3roomsroomidmessages ), calling `/messages` against a non-existent `room_id` should throw a 403 forbidden (since you're not part of the room). This also matches the behavior before https://github.com/matrix-org/synapse/pull/12370 which regressed Synapse to the 500 behavior.
```json
{
"errcode": "M_FORBIDDEN",
"error": "User @test:my.synapse.server not in room !dne:my.synapse.server, and room previews are disabled"
}
```
2022-05-10 23:39:14 -05:00
Erik Johnston
c72d26c1e1
Refactor EventContext ( #12689 )
...
Refactor how the `EventContext` class works, with the intention of reducing the amount of state we fetch from the DB during event processing.
The idea here is to get rid of the cached `current_state_ids` and `prev_state_ids` that live in the `EventContext`, and instead defer straight to the database (and its caching).
One change that may have a noticeable effect is that we now no longer prefill the `get_current_state_ids` cache on a state change. However, that query is relatively light, since its just a case of reading a table from the DB (unlike fetching state at an event which is more heavyweight). For deployments with workers this cache isn't even used.
Part of #12684
2022-05-10 19:43:13 +00:00
Sean Quah
c997bfb926
Capture the Deferred for request cancellation in _AsyncResource ( #12694 )
...
All async request processing goes through `_AsyncResource`, so this is
the only place where a `Deferred` needs to be captured for cancellation.
Unfortunately, the same isn't true for determining whether a request
can be cancelled. Each of `RestServlet`, `BaseFederationServlet`,
`DirectServe{Html,Json}Resource` and `ReplicationEndpoint` have
different wrappers around the method doing the request handling and they
all need to be handled separately.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10 20:39:05 +01:00
Patrick Cloke
29f06704b8
Fix incorrect type hint in filtering code. ( #12695 )
2022-05-10 14:10:22 -04:00
Dirk Klimpel
989fa33096
Add some type hints to datastore. ( #12477 )
2022-05-10 14:07:48 -04:00
Richard van der Hoff
147f098fb4
Stop writing to event_reference_hashes ( #12679 )
...
This table is never read, since #11794 . We stop writing to it; in future we can
drop it altogether.
2022-05-10 15:35:08 +01:00
Sean Quah
dbb12a0b54
Add helper class for testing request cancellation ( #12630 )
...
Also expose the `SynapseRequest` from `FakeChannel` in tests, so that
we can call `Request.connectionLost` to simulate a client disconnecting.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10 14:06:56 +01:00
Sean Quah
5cfb004595
Add ability to cancel disconnected requests to SynapseRequest ( #12588 )
...
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10 14:06:08 +01:00
Sean Quah
5c00151c28
Add @cancellable decorator, for use on request handlers ( #12586 )
...
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-10 14:05:22 +01:00
David Robertson
2aad0ae57f
Synapse 1.59.0rc1 (2022-05-10)
...
==============================
This release makes several changes that server administrators should be aware of:
- Device name lookup over federation is now disabled by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616 ))
- The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](https://github.com/matrix-org/synapse/issues/12452 ), [\#12654](https://github.com/matrix-org/synapse/issues/12654 ))
See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590 ) for more details.
Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](https://github.com/matrix-org/synapse/issues/12597 ))
Features
--------
- Support [MSC3266](https://github.com/matrix-org/matrix-doc/pull/3266 ) room summaries over federation. ([\#11507](https://github.com/matrix-org/synapse/issues/11507 ))
- Implement [changes](4a77139249https://github.com/matrix-org/matrix-spec-proposals/pull/2285 ). Contributed by @SimonBrandner. ([\#12168](https://github.com/matrix-org/synapse/issues/12168 ), [\#12635](https://github.com/matrix-org/synapse/issues/12635 ), [\#12636](https://github.com/matrix-org/synapse/issues/12636 ), [\#12670](https://github.com/matrix-org/synapse/issues/12670 ))
- Extend the [module API](https://github.com/matrix-org/synapse/blob/release-v1.59/synapse/module_api/__init__.py ) to allow modules to change actions for existing push rules of local users. ([\#12406](https://github.com/matrix-org/synapse/issues/12406 ))
- Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ([\#12452](https://github.com/matrix-org/synapse/issues/12452 ))
- Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. ([\#12654](https://github.com/matrix-org/synapse/issues/12654 ))
- Add new `enable_registration_token_3pid_bypass` configuration option to allow registrations via token as an alternative to verifying a 3pid. ([\#12526](https://github.com/matrix-org/synapse/issues/12526 ))
- Implement [MSC3786](https://github.com/matrix-org/matrix-spec-proposals/pull/3786 ): Add a default push rule to ignore `m.room.server_acl` events. ([\#12601](https://github.com/matrix-org/synapse/issues/12601 ))
- Add new `mau_appservice_trial_days` configuration option to specify a different trial period for users registered via an appservice. ([\#12619](https://github.com/matrix-org/synapse/issues/12619 ))
Bugfixes
--------
- Fix a bug introduced in Synapse 1.48.0 where the latest thread reply provided failed to include the proper bundled aggregations. ([\#12273](https://github.com/matrix-org/synapse/issues/12273 ))
- Fix a bug introduced in Synapse 1.22.0 where attempting to send a large amount of read receipts to an application service all at once would result in duplicate content and abnormally high memory usage. Contributed by Brad & Nick @ Beeper. ([\#12544](https://github.com/matrix-org/synapse/issues/12544 ))
- Fix a bug introduced in Synapse 1.57.0 which could cause `Failed to calculate hosts in room` errors to be logged for outbound federation. ([\#12570](https://github.com/matrix-org/synapse/issues/12570 ))
- Fix a long-standing bug where status codes would almost always get logged as `200!`, irrespective of the actual status code, when clients disconnect before a request has finished processing. ([\#12580](https://github.com/matrix-org/synapse/issues/12580 ))
- Fix race when persisting an event and deleting a room that could lead to outbound federation breaking. ([\#12594](https://github.com/matrix-org/synapse/issues/12594 ))
- Fix a bug introduced in Synapse 1.53.0 where bundled aggregations for annotations/edits were incorrectly calculated. ([\#12633](https://github.com/matrix-org/synapse/issues/12633 ))
- Fix a long-standing bug where rooms containing power levels with string values could not be upgraded. ([\#12657](https://github.com/matrix-org/synapse/issues/12657 ))
- Prevent memory leak from reoccurring when presence is disabled. ([\#12656](https://github.com/matrix-org/synapse/issues/12656 ))
Updates to the Docker image
---------------------------
- Explicitly opt-in to using [BuildKit-specific features](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md ) in the Dockerfile. This fixes issues with building images in some GitLab CI environments. ([\#12541](https://github.com/matrix-org/synapse/issues/12541 ))
- Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Contributed by @henryclw. ([\#12573](https://github.com/matrix-org/synapse/issues/12573 ))
Improved Documentation
----------------------
- Update SQL statements and replace use of old table `user_stats_historical` in docs for Synapse Admins. ([\#12536](https://github.com/matrix-org/synapse/issues/12536 ))
- Add missing linebreak to `pipx` install instructions. ([\#12579](https://github.com/matrix-org/synapse/issues/12579 ))
- Add information about the TCP replication module to docs. ([\#12621](https://github.com/matrix-org/synapse/issues/12621 ))
- Fixes to the formatting of `README.rst`. ([\#12627](https://github.com/matrix-org/synapse/issues/12627 ))
- Fix docs on how to run specific Complement tests using the `complement.sh` test runner. ([\#12664](https://github.com/matrix-org/synapse/issues/12664 ))
Deprecations and Removals
-------------------------
- Remove unstable identifiers from [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069 ). ([\#12596](https://github.com/matrix-org/synapse/issues/12596 ))
- Remove the unspecified `m.login.jwt` login type and the unstable `uk.half-shot.msc2778.login.application_service` from
[MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778 ). ([\#12597](https://github.com/matrix-org/synapse/issues/12597 ))
- Synapse now requires at least Python 3.7.1 (up from 3.7.0), for compatibility with the latest Twisted trunk. ([\#12613](https://github.com/matrix-org/synapse/issues/12613 ))
Internal Changes
----------------
- Use supervisord to supervise Postgres and Caddy in the Complement image to reduce restart time. ([\#12480](https://github.com/matrix-org/synapse/issues/12480 ))
- Immediately retry any requests that have backed off when a server comes back online. ([\#12500](https://github.com/matrix-org/synapse/issues/12500 ))
- Use `make_awaitable` instead of `defer.succeed` for return values of mocks in tests. ([\#12505](https://github.com/matrix-org/synapse/issues/12505 ))
- Consistently check if an object is a `frozendict`. ([\#12564](https://github.com/matrix-org/synapse/issues/12564 ))
- Protect module callbacks with read semantics against cancellation. ([\#12568](https://github.com/matrix-org/synapse/issues/12568 ))
- Improve comments and error messages around access tokens. ([\#12577](https://github.com/matrix-org/synapse/issues/12577 ))
- Improve docstrings for the receipts store. ([\#12581](https://github.com/matrix-org/synapse/issues/12581 ))
- Use constants for read-receipts in tests. ([\#12582](https://github.com/matrix-org/synapse/issues/12582 ))
- Log status code of cancelled requests as 499 and avoid logging stack traces for them. ([\#12587](https://github.com/matrix-org/synapse/issues/12587 ), [\#12663](https://github.com/matrix-org/synapse/issues/12663 ))
- Remove special-case for `twisted` logger from default log config. ([\#12589](https://github.com/matrix-org/synapse/issues/12589 ))
- Use `getClientAddress` instead of the deprecated `getClientIP`. ([\#12599](https://github.com/matrix-org/synapse/issues/12599 ))
- Add link to documentation in Grafana Dashboard. ([\#12602](https://github.com/matrix-org/synapse/issues/12602 ))
- Reduce log spam when running multiple event persisters. ([\#12610](https://github.com/matrix-org/synapse/issues/12610 ))
- Add extra debug logging to federation sender. ([\#12614](https://github.com/matrix-org/synapse/issues/12614 ))
- Prevent remote homeservers from requesting local user device names by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616 ))
- Add a consistency check on events which we read from the database. ([\#12620](https://github.com/matrix-org/synapse/issues/12620 ))
- Remove use of the `constantly` library and switch to enums for `EventRedactBehaviour`. Contributed by @andrewdoh. ([\#12624](https://github.com/matrix-org/synapse/issues/12624 ))
- Remove unused code related to receipts. ([\#12632](https://github.com/matrix-org/synapse/issues/12632 ))
- Minor improvements to the scripts for running Synapse in worker mode under Complement. ([\#12637](https://github.com/matrix-org/synapse/issues/12637 ))
- Move `pympler` back in to the `all` extras. ([\#12652](https://github.com/matrix-org/synapse/issues/12652 ))
- Fix spelling of `M_UNRECOGNIZED` in comments. ([\#12665](https://github.com/matrix-org/synapse/issues/12665 ))
- Release script: confirm the commit to be tagged before tagging. ([\#12556](https://github.com/matrix-org/synapse/issues/12556 ))
- Fix a typo in the announcement text generated by the Synapse release development script. ([\#12612](https://github.com/matrix-org/synapse/issues/12612 ))
- Fix scripts-dev to pass typechecking. ([\#12356](https://github.com/matrix-org/synapse/issues/12356 ))
- Add some type hints to datastore. ([\#12485](https://github.com/matrix-org/synapse/issues/12485 ))
- Remove unused `# type: ignore`s. ([\#12531](https://github.com/matrix-org/synapse/issues/12531 ))
- Allow unused `# type: ignore` comments in bleeding edge CI jobs. ([\#12576](https://github.com/matrix-org/synapse/issues/12576 ))
- Remove redundant lines of config from `mypy.ini`. ([\#12608](https://github.com/matrix-org/synapse/issues/12608 ))
- Update to mypy 0.950. ([\#12650](https://github.com/matrix-org/synapse/issues/12650 ))
- Use `Concatenate` to better annotate `_do_execute`. ([\#12666](https://github.com/matrix-org/synapse/issues/12666 ))
- Use `ParamSpec` to refine type hints. ([\#12667](https://github.com/matrix-org/synapse/issues/12667 ))
- Fix mypy against latest pillow stubs. ([\#12671](https://github.com/matrix-org/synapse/issues/12671 ))
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmJ6UDoACgkQkD7OEIo5
3t3XMw/9GfROfC///lio77EhdI0lh39bkgvfqkFE/wRYtTs4n3ZSiclgVBnNmgnN
WnLvjL/4wXRjzJZ8vcO+qG0OasT3H9l5xdGE+7h8oUMIYCG7Xdyx37xnhv5IC8Y5
dAjqs2mMnIWnAqyOAVZQVgmNLDhRAb9YGCJ3fJY9cV8dmzr8/RzR8gXrFPaBT9Nj
w5Xag0jkiJuff4+Y5oan2yBxMvHT/zAb8rQf5wNjen00lwYHSAr/5aiAsbnHcMYM
q0ZahVB/hVaA5GqxwrySlE0mqrPP6M/CpbcLr1h8LlmLdgA6f3vcf/sREGq6wCRK
Z8yYXM4RyvNTyYirQfnP5QseVazJqcCY0pCXrMnQxwXda7nSDKjXo0WFBmyIQpK4
YqDvlOhECr+Pcac/EAviH2OEvTMBg4cUs1xdm4/CBElIOZQPyXHJ0DLku8R0CRx/
i01fN0I/r/wX1Oliy4wFJgSYt/uUQzZaJmRieK8fznlFibM3Gk1GvniX0M5Hu6cZ
PQuHdPqRxQzvBmEAA1hxTfyOaDeP3g29NnOz9cjsBiAVNqUbEV5KFusjwksI4kcT
Bsdk+v/i8/Kx6t0ovRI2i3JJAFNWmUzpY0iDzY3kJCn1YmEMZ7rs+e5qRmv2eBLW
oS9dhLGsW7A/mJC6fLM+CATwbFiL/SqB5ocnl4POodZ/b27eDyw=
=NPHq
-----END PGP SIGNATURE-----
gpgsig -----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEE1508oLYUKainYFJakD7OEIo53t0FAmJ6V/YACgkQkD7OEIo5
3t0ANA/7BZ762LAjLNsFvJVxnH8LjDEjsjCyjHQ/zzVkAete3354zcW/mqN2YzMl
shaxHucfZ9sKCXlOSuwY9N+1Q24ALMAb8QebCQZjNamxKqddpMD0+ijeaKRoFq8P
6VY5saL+rjW0g2QaKE/IHc99UKnw/cNN/vGa3xc3dJFRiiNkK95jS8R7aaaT2mXB
9frbbN0Q45Q6JTU9qJTP6jqdYmKZNjvIHBJcEXkstGqS8NPJXpfVKMZIgTj/8nGL
RVJqOStsnHdx4vywZ76GPTAxQ+ZjXMilIbz+bF/6Zw1SCJEZfwFOelYApwMi0qU7
yDK7xeokKxzXyykzmptmiqmePCbr9fHbk+BJtE7VblMdj7OaM7jX7foSX6J1elIp
zwDwBK0VH3CJzk9SRoy4JW3PI9K45DZodev33CT+sT39/wYTiV3mbGIYkgP8oSDH
rd588QZDovVDy6NWzxkT0MsutHE5Rhx2aP+hkIFDHL4WWnKeFRD5HbiZfg/eC6r0
0nPRY2CCikvqLRqefTi4dgxb+twVBnioPZiDBTEKA/M1/qVg978SIpdHsfmaDJf9
BzWsvfsns0t54J2QnazOjKx2yxIYH6pZ2VFnrb47404O5O3+TuKZL9eX8oNZeBFl
JKZKD2xii/ktkw5p27RxEpBJ13hrssN+RMd0JrvS3LTa5G9aIUM=
=2gaD
-----END PGP SIGNATURE-----
Merge tag 'v1.59.0rc1' into develop
Synapse 1.59.0rc1 (2022-05-10)
==============================
This release makes several changes that server administrators should be aware of:
- Device name lookup over federation is now disabled by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616 ))
- The `synapse.app.appservice` and `synapse.app.user_dir` worker application types are now deprecated. ([\#12452](https://github.com/matrix-org/synapse/issues/12452 ), [\#12654](https://github.com/matrix-org/synapse/issues/12654 ))
See [the upgrade notes](https://github.com/matrix-org/synapse/blob/develop/docs/upgrade.md#upgrading-to-v1590 ) for more details.
Additionally, this release removes the non-standard `m.login.jwt` login type from Synapse. It can be replaced with `org.matrix.login.jwt` for identical behaviour. This is only used if `jwt_config.enabled` is set to `true` in the configuration. ([\#12597](https://github.com/matrix-org/synapse/issues/12597 ))
Features
--------
- Support [MSC3266](https://github.com/matrix-org/matrix-doc/pull/3266 ) room summaries over federation. ([\#11507](https://github.com/matrix-org/synapse/issues/11507 ))
- Implement [changes](4a77139249https://github.com/matrix-org/matrix-spec-proposals/pull/2285 ). Contributed by @SimonBrandner. ([\#12168](https://github.com/matrix-org/synapse/issues/12168 ), [\#12635](https://github.com/matrix-org/synapse/issues/12635 ), [\#12636](https://github.com/matrix-org/synapse/issues/12636 ), [\#12670](https://github.com/matrix-org/synapse/issues/12670 ))
- Extend the [module API](https://github.com/matrix-org/synapse/blob/release-v1.59/synapse/module_api/__init__.py ) to allow modules to change actions for existing push rules of local users. ([\#12406](https://github.com/matrix-org/synapse/issues/12406 ))
- Add the `notify_appservices_from_worker` configuration option (superseding `notify_appservices`) to allow a generic worker to be designated as the worker to send traffic to Application Services. ([\#12452](https://github.com/matrix-org/synapse/issues/12452 ))
- Add the `update_user_directory_from_worker` configuration option (superseding `update_user_directory`) to allow a generic worker to be designated as the worker to update the user directory. ([\#12654](https://github.com/matrix-org/synapse/issues/12654 ))
- Add new `enable_registration_token_3pid_bypass` configuration option to allow registrations via token as an alternative to verifying a 3pid. ([\#12526](https://github.com/matrix-org/synapse/issues/12526 ))
- Implement [MSC3786](https://github.com/matrix-org/matrix-spec-proposals/pull/3786 ): Add a default push rule to ignore `m.room.server_acl` events. ([\#12601](https://github.com/matrix-org/synapse/issues/12601 ))
- Add new `mau_appservice_trial_days` configuration option to specify a different trial period for users registered via an appservice. ([\#12619](https://github.com/matrix-org/synapse/issues/12619 ))
Bugfixes
--------
- Fix a bug introduced in Synapse 1.48.0 where the latest thread reply provided failed to include the proper bundled aggregations. ([\#12273](https://github.com/matrix-org/synapse/issues/12273 ))
- Fix a bug introduced in Synapse 1.22.0 where attempting to send a large amount of read receipts to an application service all at once would result in duplicate content and abnormally high memory usage. Contributed by Brad & Nick @ Beeper. ([\#12544](https://github.com/matrix-org/synapse/issues/12544 ))
- Fix a bug introduced in Synapse 1.57.0 which could cause `Failed to calculate hosts in room` errors to be logged for outbound federation. ([\#12570](https://github.com/matrix-org/synapse/issues/12570 ))
- Fix a long-standing bug where status codes would almost always get logged as `200!`, irrespective of the actual status code, when clients disconnect before a request has finished processing. ([\#12580](https://github.com/matrix-org/synapse/issues/12580 ))
- Fix race when persisting an event and deleting a room that could lead to outbound federation breaking. ([\#12594](https://github.com/matrix-org/synapse/issues/12594 ))
- Fix a bug introduced in Synapse 1.53.0 where bundled aggregations for annotations/edits were incorrectly calculated. ([\#12633](https://github.com/matrix-org/synapse/issues/12633 ))
- Fix a long-standing bug where rooms containing power levels with string values could not be upgraded. ([\#12657](https://github.com/matrix-org/synapse/issues/12657 ))
- Prevent memory leak from reoccurring when presence is disabled. ([\#12656](https://github.com/matrix-org/synapse/issues/12656 ))
Updates to the Docker image
---------------------------
- Explicitly opt-in to using [BuildKit-specific features](https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md ) in the Dockerfile. This fixes issues with building images in some GitLab CI environments. ([\#12541](https://github.com/matrix-org/synapse/issues/12541 ))
- Update the "Build docker images" GitHub Actions workflow to use `docker/metadata-action` to generate docker image tags, instead of a custom shell script. Contributed by @henryclw. ([\#12573](https://github.com/matrix-org/synapse/issues/12573 ))
Improved Documentation
----------------------
- Update SQL statements and replace use of old table `user_stats_historical` in docs for Synapse Admins. ([\#12536](https://github.com/matrix-org/synapse/issues/12536 ))
- Add missing linebreak to `pipx` install instructions. ([\#12579](https://github.com/matrix-org/synapse/issues/12579 ))
- Add information about the TCP replication module to docs. ([\#12621](https://github.com/matrix-org/synapse/issues/12621 ))
- Fixes to the formatting of `README.rst`. ([\#12627](https://github.com/matrix-org/synapse/issues/12627 ))
- Fix docs on how to run specific Complement tests using the `complement.sh` test runner. ([\#12664](https://github.com/matrix-org/synapse/issues/12664 ))
Deprecations and Removals
-------------------------
- Remove unstable identifiers from [MSC3069](https://github.com/matrix-org/matrix-doc/pull/3069 ). ([\#12596](https://github.com/matrix-org/synapse/issues/12596 ))
- Remove the unspecified `m.login.jwt` login type and the unstable `uk.half-shot.msc2778.login.application_service` from
[MSC2778](https://github.com/matrix-org/matrix-doc/pull/2778 ). ([\#12597](https://github.com/matrix-org/synapse/issues/12597 ))
- Synapse now requires at least Python 3.7.1 (up from 3.7.0), for compatibility with the latest Twisted trunk. ([\#12613](https://github.com/matrix-org/synapse/issues/12613 ))
Internal Changes
----------------
- Use supervisord to supervise Postgres and Caddy in the Complement image to reduce restart time. ([\#12480](https://github.com/matrix-org/synapse/issues/12480 ))
- Immediately retry any requests that have backed off when a server comes back online. ([\#12500](https://github.com/matrix-org/synapse/issues/12500 ))
- Use `make_awaitable` instead of `defer.succeed` for return values of mocks in tests. ([\#12505](https://github.com/matrix-org/synapse/issues/12505 ))
- Consistently check if an object is a `frozendict`. ([\#12564](https://github.com/matrix-org/synapse/issues/12564 ))
- Protect module callbacks with read semantics against cancellation. ([\#12568](https://github.com/matrix-org/synapse/issues/12568 ))
- Improve comments and error messages around access tokens. ([\#12577](https://github.com/matrix-org/synapse/issues/12577 ))
- Improve docstrings for the receipts store. ([\#12581](https://github.com/matrix-org/synapse/issues/12581 ))
- Use constants for read-receipts in tests. ([\#12582](https://github.com/matrix-org/synapse/issues/12582 ))
- Log status code of cancelled requests as 499 and avoid logging stack traces for them. ([\#12587](https://github.com/matrix-org/synapse/issues/12587 ), [\#12663](https://github.com/matrix-org/synapse/issues/12663 ))
- Remove special-case for `twisted` logger from default log config. ([\#12589](https://github.com/matrix-org/synapse/issues/12589 ))
- Use `getClientAddress` instead of the deprecated `getClientIP`. ([\#12599](https://github.com/matrix-org/synapse/issues/12599 ))
- Add link to documentation in Grafana Dashboard. ([\#12602](https://github.com/matrix-org/synapse/issues/12602 ))
- Reduce log spam when running multiple event persisters. ([\#12610](https://github.com/matrix-org/synapse/issues/12610 ))
- Add extra debug logging to federation sender. ([\#12614](https://github.com/matrix-org/synapse/issues/12614 ))
- Prevent remote homeservers from requesting local user device names by default. ([\#12616](https://github.com/matrix-org/synapse/issues/12616 ))
- Add a consistency check on events which we read from the database. ([\#12620](https://github.com/matrix-org/synapse/issues/12620 ))
- Remove use of the `constantly` library and switch to enums for `EventRedactBehaviour`. Contributed by @andrewdoh. ([\#12624](https://github.com/matrix-org/synapse/issues/12624 ))
- Remove unused code related to receipts. ([\#12632](https://github.com/matrix-org/synapse/issues/12632 ))
- Minor improvements to the scripts for running Synapse in worker mode under Complement. ([\#12637](https://github.com/matrix-org/synapse/issues/12637 ))
- Move `pympler` back in to the `all` extras. ([\#12652](https://github.com/matrix-org/synapse/issues/12652 ))
- Fix spelling of `M_UNRECOGNIZED` in comments. ([\#12665](https://github.com/matrix-org/synapse/issues/12665 ))
- Release script: confirm the commit to be tagged before tagging. ([\#12556](https://github.com/matrix-org/synapse/issues/12556 ))
- Fix a typo in the announcement text generated by the Synapse release development script. ([\#12612](https://github.com/matrix-org/synapse/issues/12612 ))
- Fix scripts-dev to pass typechecking. ([\#12356](https://github.com/matrix-org/synapse/issues/12356 ))
- Add some type hints to datastore. ([\#12485](https://github.com/matrix-org/synapse/issues/12485 ))
- Remove unused `# type: ignore`s. ([\#12531](https://github.com/matrix-org/synapse/issues/12531 ))
- Allow unused `# type: ignore` comments in bleeding edge CI jobs. ([\#12576](https://github.com/matrix-org/synapse/issues/12576 ))
- Remove redundant lines of config from `mypy.ini`. ([\#12608](https://github.com/matrix-org/synapse/issues/12608 ))
- Update to mypy 0.950. ([\#12650](https://github.com/matrix-org/synapse/issues/12650 ))
- Use `Concatenate` to better annotate `_do_execute`. ([\#12666](https://github.com/matrix-org/synapse/issues/12666 ))
- Use `ParamSpec` to refine type hints. ([\#12667](https://github.com/matrix-org/synapse/issues/12667 ))
- Fix mypy against latest pillow stubs. ([\#12671](https://github.com/matrix-org/synapse/issues/12671 ))
2022-05-10 13:17:56 +01:00
Patrick Cloke
b44fbdffa4
Move free functions into PushRuleEvaluatorForEvent. ( #12677 )
...
* Move `_condition_checker` into `PushRuleEvaluatorForEvent`.
* Move the condition cache into `PushRuleEvaluatorForEvent`.
* Improve docstrings.
* Inline a method which is only called once.
2022-05-10 07:54:30 -04:00
Patrick Cloke
02cdace707
Add class-diagrams and notes for push. ( #12676 )
2022-05-10 07:43:34 -04:00
David Robertson
efcd899f69
other fixes
2022-05-10 11:31:10 +01:00
David Robertson
735faab2b8
backquote m.room.server_acl
2022-05-10 11:30:20 +01:00
David Robertson
c707ea736a
v1 -> 1
2022-05-10 11:29:49 +01:00
David Robertson
80b3246528
Fix deprecation notice
2022-05-10 11:29:40 +01:00
David Robertson
2bae6d93c9
I manually added O's change, remove newsfile
2022-05-10 11:17:42 +01:00
David Robertson
239da21c1a
Add Olivier's last-minute merge
2022-05-10 11:12:53 +01:00
David Robertson
946b8437cf
Group release script changes
2022-05-10 11:12:53 +01:00
David Robertson
464fe99f52
Fix changelog link
2022-05-10 11:12:53 +01:00
reivilibre
699192fc1a
Add the update_user_directory_from_worker configuration option (superseding update_user_directory) to allow a generic worker to be designated as the worker to update the user directory. ( #12654 )
...
Co-authored-by: Shay <hillerys@element.io>
2022-05-10 11:08:45 +01:00
David Robertson
8ef0d85acd
Changelog typo
2022-05-10 11:07:44 +01:00
David Robertson
2cdac6f585
Adjust changelog
2022-05-10 11:06:58 +01:00
David Robertson
e5fd23fb6f
1.59.0rc1
2022-05-10 10:45:13 +01:00
Erik Johnston
8dd3e0e084
Immediately retry any requests that have backed off when a server comes back online. ( #12500 )
...
Otherwise it can take up to a minute for any in-flight `/send` requests to be retried.
2022-05-10 10:39:54 +01:00
Šimon Brandner
ade3008821
Implement MSC3786: Add a default push rule to ignore m.room.server_acl events ( #12601 )
...
Fixes vector-im/element-web#20788
Implements matrix-org/matrix-spec-proposals#3786
2022-05-10 08:57:36 +01:00
Shay
d80a7ab151
Update replication.md with info on TCP module structure ( #12621 )
2022-05-09 14:46:43 -07:00
Dirk Klimpel
615d96ad6e
Update SQL statements in docs for Synapse Admins ( #12536 )
2022-05-09 14:43:02 -07:00
Richard van der Hoff
34e84fee68
Tweaks to workers-under-complement ( #12637 )
...
* Bump the HS startup timeout
* Log prefixes for more processes
* Bump the overall timeout
2022-05-09 22:41:06 +01:00
Val Lorentz
bf0c3ca20a
Fix inconsistent spelling of 'M_UNRECOGNIZED'. ( #12665 )
2022-05-09 20:29:07 +00:00
Sean Quah
a00462dd99
Implement cancellation support/protection for module callbacks ( #12568 )
...
There's no guarantee that module callbacks will handle cancellation
appropriately. Protect module callbacks with read semantics from
cancellation and avoid swallowing `CancelledError`s that arise.
Other module callbacks, such as the `on_*` callbacks, are presumed to
live on code paths that involve writes and aren't cancellation-friendly.
These module callbacks have been left alone.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-09 12:31:14 +01:00
David Robertson
8de0facaae
Fix mypy against latest pillow stubs ( #12671 )
2022-05-09 10:48:14 +00:00
Sean Quah
41a882e62d
Update changelog for #12587 to be more accurate ( #12663 )
...
#12587 has fallen on the wrong side of the release cutoff to the rest of
the related PRs.
Signed-off-by: Sean Quah <seanq@element.io>
2022-05-09 11:34:39 +01:00
David Robertson
fa0eab9c8e
Use ParamSpec in a few places ( #12667 )
2022-05-09 10:27:39 +00:00
Erik Johnston
c5969b346d
Don't error on unknown receipt types ( #12670 )
...
Fixes #12669
2022-05-09 11:09:19 +01:00
