Commit Graph

18281 Commits

Author SHA1 Message Date
Patrick Cloke
1619802228
Various clean-ups to the logging context code (#8935) 2020-12-14 14:19:47 -05:00
Richard van der Hoff
895e04319b
Preparatory refactoring of the OidcHandlerTestCase (#8911)
* Remove references to handler._auth_handler

(and replace them with hs.get_auth_handler)

* Factor out a utility function for building Requests

* Remove mocks of `OidcHandler._map_userinfo_to_user`

This method is going away, so mocking it out is no longer a valid approach.

Instead, we mock out lower-level methods (eg _remote_id_from_userinfo), or
simply allow the regular implementation to proceed and update the expectations
accordingly.

* Remove references to `OidcHandler._map_userinfo_to_user` from tests

This method is going away, so we can no longer use it as a test point. Instead
we build mock "callback" requests which we pass into `handle_oidc_callback`,
and verify correct behaviour by mocking out `AuthHandler.complete_sso_login`.
2020-12-14 11:38:50 +00:00
David Teller
f14428b25c
Allow spam-checker modules to be provide async methods. (#8890)
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
2020-12-11 14:05:15 -05:00
Patrick Cloke
5d34f40d49
Add type hints to the push module. (#8901) 2020-12-11 11:43:53 -05:00
Erik Johnston
a8eceb01e5
Honour AS ratelimit settings for /login requests (#8920)
Fixes #8846.
2020-12-11 16:33:31 +00:00
Patrick Cloke
3af0672350
Improve tests for structured logging. (#8916) 2020-12-11 07:25:01 -05:00
Dirk Klimpel
0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Erik Johnston
1d55c7b567
Don't ratelimit autojoining of rooms (#8921)
Fixes #8866
2020-12-11 10:17:49 +00:00
Richard van der Hoff
dc016c66ae
Don't publish latest docker image until all archs are built (#8909) 2020-12-10 17:00:29 +00:00
Erik Johnston
80a992d7b9
Fix deadlock on SIGHUP (#8918)
Fixes #8892
2020-12-10 16:56:05 +00:00
Richard van der Hoff
c64002e1c1
Refactor SsoHandler.get_mxid_from_sso (#8900)
* Factor out _call_attribute_mapper and _register_mapped_user

This is mostly an attempt to simplify `get_mxid_from_sso`.

* Move mapping_lock down into SsoHandler.
2020-12-10 12:43:58 +00:00
Richard van der Hoff
1821f7cc26
Fix buglet in DirectRenderJsonResource (#8897)
this was using `canonical_json` without setting it, so when you used it as a
standalone class, you would get exceptions.
2020-12-10 12:42:55 +00:00
Dirk Klimpel
a5f7aff5e5
Deprecate Shutdown Room and Purge Room Admin API (#8829)
Deprecate both APIs in favour of the Delete Room API.

Related: #8663 and #8810
2020-12-10 11:42:48 +00:00
Patrick Cloke
344ab0b53a
Default to blacklisting reserved IP ranges and add a whitelist. (#8870)
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
2020-12-09 13:56:06 -05:00
Patrick Cloke
6ff34e00d9
Skip the SAML tests if xmlsec1 isn't available. (#8905) 2020-12-09 12:23:30 -05:00
Dirk Klimpel
43bf3c5178
Combine related media admin API docs (#8839)
Related: #8810
Also a few small improvements.

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-12-09 16:19:57 +00:00
Richard van der Hoff
a4a5c7a35e Merge remote-tracking branch 'origin/master' into develop 2020-12-09 16:13:52 +00:00
Richard van der Hoff
3e8292d483
Merge pull request #8906 from matrix-org/rav/fix_multiarch_builds
Pin the docker version for multiarch builds
2020-12-09 16:03:12 +00:00
Erik Johnston
cf7d3c90d6 Merge branch 'release-v1.24.0' into develop 2020-12-09 16:01:12 +00:00
Richard van der Hoff
9bbbb11ac2 Pin the docker version for multiarch builds
It seems that letting CircleCI use its default docker version (17.09.0-ce,
apparently) did not interact well with multiarch builds: in particular, we saw
weird effects where running an amd64 build at the same time as an arm64 build
caused the arm64 builds to fail with:

   Error while loading /usr/sbin/dpkg-deb: No such file or directory
2020-12-09 15:51:11 +00:00
Erik Johnston
57068eae75 Add 'xmlsec1' to dependency list 2020-12-09 13:48:16 +00:00
Erik Johnston
fd83debcc0 Merge branch 'master' into develop 2020-12-09 11:30:08 +00:00
Erik Johnston
320e8c8064 Synapse 1.23.1 (2020-12-09)
===========================
 
 Due to the two security issues highlighted below, server administrators are
 encouraged to update Synapse. We are not aware of these vulnerabilities being
 exploited in the wild.
 
 Security advisory
 -----------------
 
 The following issues are fixed in v1.23.1 and v1.24.0.
 
 - There is a denial of service attack
   ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
   against the federation APIs in which future events will not be correctly sent
   to other servers over federation. This affects all servers that participate in
   open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).
 
 - Synapse may be affected by OpenSSL
   [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
   Synapse administrators should ensure that they have the latest versions of
   the cryptography Python package installed.
 
 To upgrade Synapse along with the cryptography package:
 
 * Administrators using the [`matrix.org` Docker
   image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
   packages from
   `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
   should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
   the updated packages.
 * Administrators who have [installed Synapse from
   source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
   should upgrade the cryptography package within their virtualenv by running:
   ```sh
   <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
   ```
 * Administrators who have installed Synapse from distribution packages should
   consult the information from their distributions.
 
 Bugfixes
 --------
 
 - Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))
 
 Internal Changes
 ----------------
 
 - Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAl/QsOYQHGVyaWtAbWF0
 cml4Lm9yZwAKCRClQuTtGw+sCZTkCACEDbyMY/UCqJaUILxtYeBE7K4GvOqPPHyo
 2VLjyitI7XWVzB/paUOPxAtOtiwXS0GOrL+UsW6Lky2HIjafjLe1Z3LHzATQwF2I
 J2bZWTY1Y4v3y8B7noPmp7+QFIBIey++09BY+MwzT3EQYnXt6lvoHmEvPH/htzjg
 LfdZpSj4WrJr4S2/W0rVlkGSuIShN0Tnv6pTgbGRZMt1N4JH2mo65mCGt3xrMS7E
 us+xqStGh5Q+9g3F913iIJ8noUMeCvTT7hbr1eonhZ3MIKWG30z+zcXwmGb0t3B8
 zvTFXqdbZPSw+ZZdxaZwZuJzNCnYOu6t0JuzXqDoE0xsHb8RVUe9
 =Z9US
 -----END PGP SIGNATURE-----

Merge tag 'v1.23.1'

Synapse 1.23.1 (2020-12-09)
===========================

Due to the two security issues highlighted below, server administrators are
encouraged to update Synapse. We are not aware of these vulnerabilities being
exploited in the wild.

Security advisory
-----------------

The following issues are fixed in v1.23.1 and v1.24.0.

- There is a denial of service attack
  ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
  against the federation APIs in which future events will not be correctly sent
  to other servers over federation. This affects all servers that participate in
  open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).

- Synapse may be affected by OpenSSL
  [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
  Synapse administrators should ensure that they have the latest versions of
  the cryptography Python package installed.

To upgrade Synapse along with the cryptography package:

* Administrators using the [`matrix.org` Docker
  image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
  packages from
  `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
  should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
  the updated packages.
* Administrators who have [installed Synapse from
  source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
  should upgrade the cryptography package within their virtualenv by running:
  ```sh
  <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
  ```
* Administrators who have installed Synapse from distribution packages should
  consult the information from their distributions.

Bugfixes
--------

- Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))

Internal Changes
----------------

- Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
2020-12-09 11:29:56 +00:00
Erik Johnston
adfc9cb53d Merge branch 'master' into develop 2020-12-09 11:26:48 +00:00
Erik Johnston
1cec3d1457 1.23.1 2020-12-09 11:07:41 +00:00
Erik Johnston
9b26a4ac87 1.24.0 2020-12-09 11:07:24 +00:00
Patrick Cloke
0eb9b2f866 Fix installing pysaml2 on Python 3.5. (#8898)
This pins pysaml2 to < 6.4.0 on Python 3.5, as the last known working version.
2020-12-09 10:38:46 +00:00
Richard van der Hoff
3ce2f303f1 Consistently use room_id from federation request body (#8776)
* Consistently use room_id from federation request body

Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.

* Kill off some references to "context"

Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-12-09 10:38:39 +00:00
Aaron Raimist
cd9e72b185
Add X-Robots-Tag header to stop crawlers from indexing media (#8887)
Fixes / related to: https://github.com/matrix-org/synapse/issues/6533

This should do essentially the same thing as a robots.txt file telling robots to not index the media repo. https://developers.google.com/search/reference/robots_meta_tag

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-12-08 22:51:03 +00:00
Patrick Cloke
1a9553045c
Fix installing pysaml2 on Python 3.5. (#8898)
This pins pysaml2 to < 6.4.0 on Python 3.5, as the last known working version.
2020-12-08 13:41:25 -05:00
Richard van der Hoff
ab7a24cc6b
Better formatting for config errors from modules (#8874)
The idea is that the parse_config method of extension modules can raise either a ConfigError or a JsonValidationError,
and it will be magically turned into a legible error message. There's a few components to it:

* Separating the "path" and the "message" parts of a ConfigError, so that we can fiddle with the path bit to turn it
   into an absolute path.
* Generally improving the way ConfigErrors get printed.
* Passing in the config path to load_module so that it can wrap any exceptions that get caught appropriately.
2020-12-08 14:04:35 +00:00
Richard van der Hoff
36ba73f53d
Simplify the flow for SSO UIA (#8881)
* SsoHandler: remove inheritance from BaseHandler

* Simplify the flow for SSO UIA

We don't need to do all the magic for mapping users when we are doing UIA, so
let's factor that out.
2020-12-08 14:03:38 +00:00
Richard van der Hoff
025fa06fc7
Clarify config template comments (#8891) 2020-12-08 14:03:08 +00:00
Will Hunt
ff1f0ee094
Call set_avatar_url with target_user, not user_id (#8872)
* Call set_avatar_url with target_user, not user_id

Fixes https://github.com/matrix-org/synapse/issues/8871

* Create 8872.bugfix

* Update synapse/rest/admin/users.py

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>

* Testing

* Update changelog.d/8872.bugfix

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>

Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2020-12-07 19:13:07 +00:00
Patrick Cloke
1f3748f033
Do not raise a 500 exception when previewing empty media. (#8883) 2020-12-07 10:00:08 -05:00
Patrick Cloke
92d87c6882
Add type hints for HTTP and email pushers. (#8880) 2020-12-07 09:59:38 -05:00
Patrick Cloke
02e588856a
Add type hints to the push mailer module. (#8882) 2020-12-07 07:10:22 -05:00
Patrick Cloke
96358cb424
Add authentication to replication endpoints. (#8853)
Authentication is done by checking a shared secret provided
in the Synapse configuration file.
2020-12-04 10:56:28 -05:00
Erik Johnston
df4b1e9c74
Pass room_id to get_auth_chain_difference (#8879)
This is so that we can choose which algorithm to use based on the room ID.
2020-12-04 15:52:49 +00:00
Patrick Cloke
b774c555d8
Add additional validation to pusher URLs. (#8865)
Pusher URLs now must end in `/_matrix/push/v1/notify` per the
specification.
2020-12-04 10:51:56 -05:00
Patrick Cloke
df3e6a23a7
Do not 500 if the content-length is not provided when uploading media. (#8862)
Instead return the proper 400 error.
2020-12-04 10:26:09 -05:00
Patrick Cloke
112f6bd49e Synapse 1.24.0rc2 (2020-12-04)
==============================
 
 Bugfixes
 --------
 
 - Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))
 
 Internal Changes
 ----------------
 
 - Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl/KQ6MACgkQM/xY9qcR
 MEjqOg/9Fd4Ow0QdnK3O6J0bmXP6+g2GfjJCAKlYlF/nwX1ez9jZVed00D2h9/hD
 YCUJCEv+XZpbv5F4usiYwoLGithbueo7AKEOQJQMEuQBHU1/E6TJ9iiKR3lTBNNw
 ccuyMMqqEatYxoa4DIO/lrwm2mhwRNdMPt0r1DPlML13bb/TVbeXXykbOCyZSKm1
 YuXpxYDngRPL9o46I77l7/KNTSpYYeZi2qwp2orl6GBfa0KSvR2Oi6uohKYzuoHC
 +XLvHSFMpHAZkaZsIMxU5hRVH6jtkb/D37CABuKQsdlvHcooGK4Sdt6aoUyhPtoh
 6YwajSazWKEJ/9YO5JV22qkWFPi9T6s/lPZfjOSu/euwMw7DDQJ/8t8Sm6N7sJDS
 A/5FctOAd1jd9BiAaqkHB2zLHZSomIHQpJJfDyE3UgAGQQ6fm2Wg7SkLByDx/MYC
 4D74dBuVBPENM2VxNoyjHbpVSeQi8t0RgWb44/PUAOtpwm3f3eVRuS8zY6uR2bZr
 YZUbJylqKhRW6dBdZH+EzVolRGvBgLkc27IzRCQ3kGUEo2Wem05s6MuKMvLxoIvN
 NHLWNClUWITT8FgkfFP3c6CK40dwIGJzYryI+71qPW/R7snf2fazuoHlH1pAmFh0
 E0/nBMTw1deixOYXbG1ah63AhJ1NjdTfimDsKmWZLqVc6g7g4d4=
 =is1Y
 -----END PGP SIGNATURE-----

Merge tag 'v1.24.0rc2' into develop

Synapse 1.24.0rc2 (2020-12-04)
==============================

Bugfixes
--------

- Fix a regression in v1.24.0rc1 which failed to allow SAML mapping providers which were unable to redirect users to an additional page. ([\#8878](https://github.com/matrix-org/synapse/issues/8878))

Internal Changes
----------------

- Add support for the `prometheus_client` newer than 0.9.0. Contributed by Jordan Bancino. ([\#8875](https://github.com/matrix-org/synapse/issues/8875))
2020-12-04 09:14:31 -05:00
Patrick Cloke
2602514f34 Minor update to CHANGES. 2020-12-04 09:00:32 -05:00
Patrick Cloke
693dab487c 1.24.0rc2 2020-12-04 08:48:04 -05:00
Patrick Cloke
22c6c19f91
Fix a regression that mapping providers should be able to redirect users. (#8878)
This was broken in #8801.
2020-12-04 08:25:15 -05:00
Jordan Bancino
295c209cdd
Remove version pin prometheus_client dependency (#8875)
This removes the version pin of the `prometheus_client` dependency, in direct response to #8831. If merged, this will close #8831 

As far as I can tell, no other changes are needed, but as I'm no synapse expert, I'm relying heavily on CI and maintainer reviews for this. My very primitive test of synapse with prometheus_client v0.9.0 on my home server didn't bring up any issues, so we'll see what happens.

Signed-off-by: Jordan Bancino
2020-12-04 13:01:06 +00:00
Richard van der Hoff
6e4f71c057
Fix a buglet in the SAML username mapping provider doc (#8873)
the constructor is called with a `module_api`.
2020-12-04 10:14:15 +00:00
Richard van der Hoff
cf3b8156be
Fix errorcode for disabled registration (#8867)
The spec says we should return `M_FORBIDDEN` when someone tries to register and
registration is disabled.
2020-12-03 15:41:19 +00:00
Richard van der Hoff
66f75c5b74
Merge pull request #8861 from matrix-org/rav/remove_unused_mocks
Remove some unnecessary mocking from the unit tests
2020-12-03 10:02:47 +00:00
Richard van der Hoff
269ba1bc84 Merge remote-tracking branch 'origin/develop' into rav/remove_unused_mocks 2020-12-02 20:08:46 +00:00