Commit Graph

18350 Commits

Author SHA1 Message Date
Patrick Cloke
bd30cfe86a
Convert internal pusher dicts to attrs classes. (#8940)
This improves type hinting and should use less memory.
2020-12-16 11:25:30 -05:00
Richard van der Hoff
7a332850e6
Merge pull request #8951 from matrix-org/rav/username_picker_2
More preparatory refactoring of the OidcHandler tests
2020-12-16 14:53:26 +00:00
Richard van der Hoff
651e1ae534
Merge pull request #8946 from matrix-org/rav/refactor_send_request
Remove `Request` return value from `make_request`
2020-12-16 14:53:01 +00:00
Richard van der Hoff
3ad699cc65
Fix generate_log_config script (#8952)
It used to write an empty file if you gave it a -o arg.
2020-12-16 14:52:04 +00:00
Patrick Cloke
be2db93b3c
Do not assume that the contents dictionary includes history_visibility. (#8945) 2020-12-16 08:46:37 -05:00
Richard van der Hoff
757b5a0bf6 changelog 2020-12-15 23:11:42 +00:00
Richard van der Hoff
8388a7fb3a Make _make_callback_with_userinfo async
... so that we can test its behaviour when it raises.

Also pull it out to the top level so that I can use it from other test classes.
2020-12-15 23:10:59 +00:00
Richard van der Hoff
c1883f042d Remove spurious mocking of complete_sso_login
The tests that need this all do it already.
2020-12-15 23:10:59 +00:00
Richard van der Hoff
2dd2e90e2b Test get_extra_attributes fallback
despite the warnings saying "don't implement get_extra_attributes", we had
implemented it, so the tests weren't doing what we thought they were.
2020-12-15 23:10:59 +00:00
Richard van der Hoff
c9dd47d668 lint 2020-12-15 22:35:50 +00:00
Richard van der Hoff
ed61fe4ada changelog 2020-12-15 22:35:50 +00:00
Richard van der Hoff
394516ad1b Remove spurious "SynapseRequest" result from `make_request"
This was never used, so let's get rid of it.
2020-12-15 22:35:40 +00:00
Richard van der Hoff
ac2acf1524 Remove redundant reading of SynapseRequest.args
this didn't seem to be doing a lot, so remove it.
2020-12-15 22:35:03 +00:00
Richard van der Hoff
5bcf6e8289 Skip redundant check on request.args 2020-12-15 22:35:03 +00:00
Richard van der Hoff
0378581c13 remove 'response' result from _get_shared_rooms 2020-12-15 22:34:20 +00:00
Richard van der Hoff
7eebe4b3fc Replace request.code with channel.code
The two are equivalent, but really we want to check the HTTP result that got
returned to the channel, not the code that the Request object *intended* to
return to the channel.
2020-12-15 22:32:12 +00:00
Richard van der Hoff
01333681bc
Preparatory refactoring of the SamlHandlerTestCase (#8938)
* move simple_async_mock to test_utils

... so that it can be re-used

* Remove references to `SamlHandler._map_saml_response_to_user` from tests

This method is going away, so we can no longer use it as a test point. Instead,
factor out a higher-level method which takes a SAML object, and verify correct
behaviour by mocking out `AuthHandler.complete_sso_login`.

* changelog
2020-12-15 20:56:10 +00:00
Patrick Cloke
b3a4b53587
Fix handling of stream tokens for push. (#8943)
Removes faulty assertions and fixes the logic to ensure the max
stream token is always set.
2020-12-15 10:41:34 -05:00
Richard van der Hoff
6d02eb22df
Fix startup failure with localdb_enabled: False (#8937) 2020-12-14 20:42:03 +00:00
Patrick Cloke
1619802228
Various clean-ups to the logging context code (#8935) 2020-12-14 14:19:47 -05:00
Richard van der Hoff
895e04319b
Preparatory refactoring of the OidcHandlerTestCase (#8911)
* Remove references to handler._auth_handler

(and replace them with hs.get_auth_handler)

* Factor out a utility function for building Requests

* Remove mocks of `OidcHandler._map_userinfo_to_user`

This method is going away, so mocking it out is no longer a valid approach.

Instead, we mock out lower-level methods (eg _remote_id_from_userinfo), or
simply allow the regular implementation to proceed and update the expectations
accordingly.

* Remove references to `OidcHandler._map_userinfo_to_user` from tests

This method is going away, so we can no longer use it as a test point. Instead
we build mock "callback" requests which we pass into `handle_oidc_callback`,
and verify correct behaviour by mocking out `AuthHandler.complete_sso_login`.
2020-12-14 11:38:50 +00:00
David Teller
f14428b25c
Allow spam-checker modules to be provide async methods. (#8890)
Spam checker modules can now provide async methods. This is implemented
in a backwards-compatible manner.
2020-12-11 14:05:15 -05:00
Patrick Cloke
5d34f40d49
Add type hints to the push module. (#8901) 2020-12-11 11:43:53 -05:00
Erik Johnston
a8eceb01e5
Honour AS ratelimit settings for /login requests (#8920)
Fixes #8846.
2020-12-11 16:33:31 +00:00
Patrick Cloke
3af0672350
Improve tests for structured logging. (#8916) 2020-12-11 07:25:01 -05:00
Dirk Klimpel
0a34cdfc66
Add number of local devices to Room Details Admin API (#8886) 2020-12-11 10:42:47 +00:00
Erik Johnston
1d55c7b567
Don't ratelimit autojoining of rooms (#8921)
Fixes #8866
2020-12-11 10:17:49 +00:00
Richard van der Hoff
dc016c66ae
Don't publish latest docker image until all archs are built (#8909) 2020-12-10 17:00:29 +00:00
Erik Johnston
80a992d7b9
Fix deadlock on SIGHUP (#8918)
Fixes #8892
2020-12-10 16:56:05 +00:00
Richard van der Hoff
c64002e1c1
Refactor SsoHandler.get_mxid_from_sso (#8900)
* Factor out _call_attribute_mapper and _register_mapped_user

This is mostly an attempt to simplify `get_mxid_from_sso`.

* Move mapping_lock down into SsoHandler.
2020-12-10 12:43:58 +00:00
Richard van der Hoff
1821f7cc26
Fix buglet in DirectRenderJsonResource (#8897)
this was using `canonical_json` without setting it, so when you used it as a
standalone class, you would get exceptions.
2020-12-10 12:42:55 +00:00
Dirk Klimpel
a5f7aff5e5
Deprecate Shutdown Room and Purge Room Admin API (#8829)
Deprecate both APIs in favour of the Delete Room API.

Related: #8663 and #8810
2020-12-10 11:42:48 +00:00
Patrick Cloke
344ab0b53a
Default to blacklisting reserved IP ranges and add a whitelist. (#8870)
This defaults `ip_range_blacklist` to reserved IP ranges and also adds an
`ip_range_whitelist` setting to override it.
2020-12-09 13:56:06 -05:00
Patrick Cloke
6ff34e00d9
Skip the SAML tests if xmlsec1 isn't available. (#8905) 2020-12-09 12:23:30 -05:00
Dirk Klimpel
43bf3c5178
Combine related media admin API docs (#8839)
Related: #8810
Also a few small improvements.

Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-12-09 16:19:57 +00:00
Richard van der Hoff
a4a5c7a35e Merge remote-tracking branch 'origin/master' into develop 2020-12-09 16:13:52 +00:00
Richard van der Hoff
3e8292d483
Merge pull request #8906 from matrix-org/rav/fix_multiarch_builds
Pin the docker version for multiarch builds
2020-12-09 16:03:12 +00:00
Erik Johnston
cf7d3c90d6 Merge branch 'release-v1.24.0' into develop 2020-12-09 16:01:12 +00:00
Richard van der Hoff
9bbbb11ac2 Pin the docker version for multiarch builds
It seems that letting CircleCI use its default docker version (17.09.0-ce,
apparently) did not interact well with multiarch builds: in particular, we saw
weird effects where running an amd64 build at the same time as an arm64 build
caused the arm64 builds to fail with:

   Error while loading /usr/sbin/dpkg-deb: No such file or directory
2020-12-09 15:51:11 +00:00
Erik Johnston
57068eae75 Add 'xmlsec1' to dependency list 2020-12-09 13:48:16 +00:00
Erik Johnston
fd83debcc0 Merge branch 'master' into develop 2020-12-09 11:30:08 +00:00
Erik Johnston
320e8c8064 Synapse 1.23.1 (2020-12-09)
===========================
 
 Due to the two security issues highlighted below, server administrators are
 encouraged to update Synapse. We are not aware of these vulnerabilities being
 exploited in the wild.
 
 Security advisory
 -----------------
 
 The following issues are fixed in v1.23.1 and v1.24.0.
 
 - There is a denial of service attack
   ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
   against the federation APIs in which future events will not be correctly sent
   to other servers over federation. This affects all servers that participate in
   open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).
 
 - Synapse may be affected by OpenSSL
   [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
   Synapse administrators should ensure that they have the latest versions of
   the cryptography Python package installed.
 
 To upgrade Synapse along with the cryptography package:
 
 * Administrators using the [`matrix.org` Docker
   image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
   packages from
   `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
   should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
   the updated packages.
 * Administrators who have [installed Synapse from
   source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
   should upgrade the cryptography package within their virtualenv by running:
   ```sh
   <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
   ```
 * Administrators who have installed Synapse from distribution packages should
   consult the information from their distributions.
 
 Bugfixes
 --------
 
 - Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))
 
 Internal Changes
 ----------------
 
 - Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
 -----BEGIN PGP SIGNATURE-----
 
 iQFEBAABCgAuFiEEBTGR3/RnAzBGUif3pULk7RsPrAkFAl/QsOYQHGVyaWtAbWF0
 cml4Lm9yZwAKCRClQuTtGw+sCZTkCACEDbyMY/UCqJaUILxtYeBE7K4GvOqPPHyo
 2VLjyitI7XWVzB/paUOPxAtOtiwXS0GOrL+UsW6Lky2HIjafjLe1Z3LHzATQwF2I
 J2bZWTY1Y4v3y8B7noPmp7+QFIBIey++09BY+MwzT3EQYnXt6lvoHmEvPH/htzjg
 LfdZpSj4WrJr4S2/W0rVlkGSuIShN0Tnv6pTgbGRZMt1N4JH2mo65mCGt3xrMS7E
 us+xqStGh5Q+9g3F913iIJ8noUMeCvTT7hbr1eonhZ3MIKWG30z+zcXwmGb0t3B8
 zvTFXqdbZPSw+ZZdxaZwZuJzNCnYOu6t0JuzXqDoE0xsHb8RVUe9
 =Z9US
 -----END PGP SIGNATURE-----

Merge tag 'v1.23.1'

Synapse 1.23.1 (2020-12-09)
===========================

Due to the two security issues highlighted below, server administrators are
encouraged to update Synapse. We are not aware of these vulnerabilities being
exploited in the wild.

Security advisory
-----------------

The following issues are fixed in v1.23.1 and v1.24.0.

- There is a denial of service attack
  ([CVE-2020-26257](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-26257))
  against the federation APIs in which future events will not be correctly sent
  to other servers over federation. This affects all servers that participate in
  open federation. (Fixed in [#8776](https://github.com/matrix-org/synapse/pull/8776)).

- Synapse may be affected by OpenSSL
  [CVE-2020-1971](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-1971).
  Synapse administrators should ensure that they have the latest versions of
  the cryptography Python package installed.

To upgrade Synapse along with the cryptography package:

* Administrators using the [`matrix.org` Docker
  image](https://hub.docker.com/r/matrixdotorg/synapse/) or the [Debian/Ubuntu
  packages from
  `matrix.org`](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#matrixorg-packages)
  should ensure that they have version 1.24.0 or 1.23.1 installed: these images include
  the updated packages.
* Administrators who have [installed Synapse from
  source](https://github.com/matrix-org/synapse/blob/master/INSTALL.md#installing-from-source)
  should upgrade the cryptography package within their virtualenv by running:
  ```sh
  <path_to_virtualenv>/bin/pip install 'cryptography>=3.3'
  ```
* Administrators who have installed Synapse from distribution packages should
  consult the information from their distributions.

Bugfixes
--------

- Fix a bug in some federation APIs which could lead to unexpected behaviour if different parameters were set in the URI and the request body. ([\#8776](https://github.com/matrix-org/synapse/issues/8776))

Internal Changes
----------------

- Add a maximum version for pysaml2 on Python 3.5. ([\#8898](https://github.com/matrix-org/synapse/issues/8898))
2020-12-09 11:29:56 +00:00
Erik Johnston
adfc9cb53d Merge branch 'master' into develop 2020-12-09 11:26:48 +00:00
Erik Johnston
1cec3d1457 1.23.1 2020-12-09 11:07:41 +00:00
Erik Johnston
9b26a4ac87 1.24.0 2020-12-09 11:07:24 +00:00
Patrick Cloke
0eb9b2f866 Fix installing pysaml2 on Python 3.5. (#8898)
This pins pysaml2 to < 6.4.0 on Python 3.5, as the last known working version.
2020-12-09 10:38:46 +00:00
Richard van der Hoff
3ce2f303f1 Consistently use room_id from federation request body (#8776)
* Consistently use room_id from federation request body

Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.

* Kill off some references to "context"

Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-12-09 10:38:39 +00:00
Aaron Raimist
cd9e72b185
Add X-Robots-Tag header to stop crawlers from indexing media (#8887)
Fixes / related to: https://github.com/matrix-org/synapse/issues/6533

This should do essentially the same thing as a robots.txt file telling robots to not index the media repo. https://developers.google.com/search/reference/robots_meta_tag

Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-12-08 22:51:03 +00:00
Patrick Cloke
1a9553045c
Fix installing pysaml2 on Python 3.5. (#8898)
This pins pysaml2 to < 6.4.0 on Python 3.5, as the last known working version.
2020-12-08 13:41:25 -05:00
Richard van der Hoff
ab7a24cc6b
Better formatting for config errors from modules (#8874)
The idea is that the parse_config method of extension modules can raise either a ConfigError or a JsonValidationError,
and it will be magically turned into a legible error message. There's a few components to it:

* Separating the "path" and the "message" parts of a ConfigError, so that we can fiddle with the path bit to turn it
   into an absolute path.
* Generally improving the way ConfigErrors get printed.
* Passing in the config path to load_module so that it can wrap any exceptions that get caught appropriately.
2020-12-08 14:04:35 +00:00