Commit Graph

18122 Commits

Author SHA1 Message Date
Richard van der Hoff
7ea85302f3 fix up various test cases
A few test cases were relying on being able to mount non-client servlets on the
test resource. it's better to give them their own Resources.
2020-12-02 16:30:01 +00:00
Patrick Cloke
30fba62108
Apply an IP range blacklist to push and key revocation requests. (#8821)
Replaces the `federation_ip_range_blacklist` configuration setting with an
`ip_range_blacklist` setting with wider scope. It now applies to:

* Federation
* Identity servers
* Push notifications
* Checking key validitity for third-party invite events

The old `federation_ip_range_blacklist` setting is still honored if present, but
with reduced scope (it only applies to federation and identity servers).
2020-12-02 11:09:24 -05:00
Erik Johnston
c5b6abd53d
Correctly handle unpersisted events when calculating auth chain difference. (#8827)
We do state res with unpersisted events when calculating the new current state of the room, so that should be the only thing impacted. I don't think this is tooooo big of a deal as:

1. the next time a state event happens in the room the current state should correct itself;
2. in the common case all the unpersisted events' auth events will be pulled in by other state, so will still return the correct result (or one which is sufficiently close to not affect the result); and
3. we mostly use the state at an event to do important operations, which isn't affected by this.
2020-12-02 15:22:37 +00:00
Richard van der Hoff
693516e756 Add create_resource_dict method to HomeserverTestCase
Rather than using a single JsonResource, construct a resource tree, as we do in
the prod code, and allow testcases to add extra resources by overriding
`create_resource_dict`.
2020-12-02 15:21:00 +00:00
Johanna Dorothea Reichmann
0fed46ebe5
Add missing prometheus rules for persisted events (#8802)
The official dashboard uses data from these rules, but they were never added to the synapse-v2.rules. They are mentioned in this issue: https://github.com/matrix-org/synapse/issues/7917#issuecomment-661330409, but never got added to the rules.

Adding them results in all graphs in the "Event persist rate" section to function as intended.

Signed-off-by: Johanna Dorothea Reichmann <transcaffeine@finallycoffee.eu>
2020-12-02 15:18:41 +00:00
David Florness
c4675e1b24
Add additional validation for the admin register endpoint. (#8837)
Raise a proper 400 error if the `mac` field is missing.
2020-12-02 10:01:15 -05:00
Patrick Cloke
e41720d85f Minor changes to the CHANGES doc. 2020-12-02 09:17:42 -05:00
Patrick Cloke
c67af840aa Minor fixes to changelog. 2020-12-02 09:03:12 -05:00
Patrick Cloke
53b12688dd 1.24.0rc1 2020-12-02 08:57:51 -05:00
Patrick Cloke
8388384a64
Fix a regression when grandfathering SAML users. (#8855)
This was broken in #8801 when abstracting code shared with OIDC.

After this change both SAML and OIDC have a concept of
grandfathering users, but with different implementations.
2020-12-02 07:45:42 -05:00
Patrick Cloke
c21bdc813f
Add basic SAML tests for mapping users. (#8800) 2020-12-02 07:09:21 -05:00
Richard van der Hoff
d3ed93504b
Create a PasswordProvider wrapper object (#8849)
The idea here is to abstract out all the conditional code which tests which
methods a given password provider has, to provide a consistent interface.
2020-12-02 10:38:50 +00:00
Andrew Morgan
edb3d3f827
Allow specifying room version in 'RestHelper.create_room_as' and add typing (#8854)
This PR adds a `room_version` argument to the `RestHelper`'s `create_room_as` function for tests. I plan to use this for testing knocking, which currently uses an unstable room version.
2020-12-02 10:38:18 +00:00
Richard van der Hoff
4d9496559d
Support "identifier" dicts in UIA (#8848)
The spec requires synapse to support `identifier` dicts for `m.login.password`
user-interactive auth, which it did not (instead, it required an undocumented
`user` parameter.)

To fix this properly, we need to pull the code that interprets `identifier`
into `AuthHandler.validate_login` so that it can be called from the UIA code.

Fixes #5665.
2020-12-01 17:42:26 +00:00
Richard van der Hoff
9edff901d1
Add missing ordering to background updates (#8850)
It's important that we make sure our background updates happen in a defined
order, to avoid disasters like #6923.

Add an ordering to all of the background updates that have landed since #7190.
2020-12-01 15:52:49 +00:00
Nicolas Chamo
3f0cba657c
Allow Date header through CORS (#8804) 2020-12-01 13:24:56 +00:00
Richard van der Hoff
89f7930730
Don't offer password login when it is disabled (#8835)
Fix a minor bug where we would offer "m.login.password" login if a custom auth provider supported it, even if password login was disabled.
2020-12-01 13:04:03 +00:00
Richard van der Hoff
ddc4343683
Add some tests for password_auth_providers (#8819)
These things seemed to be completely untested, so I added a load of tests for
them.
2020-12-01 11:10:42 +00:00
Richard van der Hoff
09ac0569fe
Fix broken testcase (#8851)
This test was broken by #8565. It doesn't need to set set `self.clock`
here anyway - that is done by `setUp`.
2020-12-01 11:04:57 +00:00
Andrew Morgan
d1be293f00 Fix typo in password_auth_providers doc
A word got removed accidentally in 83434df381.
2020-12-01 10:34:52 +00:00
Richard van der Hoff
59e18a1333
Simplify appservice login code (#8847)
we don't need to support legacy login dictionaries here.
2020-11-30 19:20:56 +00:00
Mathieu Velten
9f0f274fe0
Allow per-room profile to be used for server notice user (#8799)
This applies even if the feature is disabled at the server level with `allow_per_room_profiles`.
The server notice not being a real user it doesn't have an user profile.
2020-11-30 18:59:29 +00:00
Richard van der Hoff
f8d13ca13d
Drop (almost) unused index on event_json (#8845) 2020-11-30 18:44:09 +00:00
Andrew Morgan
17fa58bdd1
Add a config option to change whether unread push notification counts are per-message or per-room (#8820)
This PR adds a new config option to the `push` section of the homeserver config, `group_unread_count_by_room`. By default Synapse will group push notifications by room (so if you have 1000 unread messages, if they lie in 55 rooms, you'll see an unread count on your phone of 55).

However, it is also useful to be able to send out the true count of unread messages if desired. If `group_unread_count_by_room` is set to `false`, then with the above example, one would see an unread count of 1000 (email anyone?).
2020-11-30 18:43:54 +00:00
Jonathan de Jong
ca60822b34
Simplify the way the HomeServer object caches its internal attributes. (#8565)
Changes `@cache_in_self` to use underscore-prefixed attributes.
2020-11-30 13:28:44 -05:00
Richard van der Hoff
a090b86209
Add force_purge option to delete-room admin api. (#8843) 2020-11-30 16:48:12 +00:00
Tulir Asokan
856eab606b
Remove special case of pretty printing JSON responses for curl (#8833)
* Remove special case of pretty printing JSON responses for curl

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2020-11-27 12:37:55 +00:00
Andrew Morgan
5cbe8d93fe
Add typing to membership Replication class methods (#8809)
This PR grew out of #6739, and adds typing to some method arguments

You'll notice that there are a lot of `# type: ignores` in here. This is due to the base methods not matching the overloads here. This is necessary to stop mypy complaining, but a better solution is #8828.
2020-11-27 10:49:38 +00:00
Dirk Klimpel
1cd356765e
Update example prometheus console (#8824)
Signed-off-by: Dirk Klimpel dirk@klimpel.org
2020-11-26 17:41:20 +00:00
Erik Johnston
382b4e83f1
Defer SIGHUP handlers to reactor. (#8817)
We can get a SIGHUP at any point, including times where we are not in a
sane state. By deferring calling the handlers until the next reactor
tick we ensure that we don't get unexpected conflicts, e.g. trying to
flush logs from the signal handler while the code was in the process of
writing a log entry.

Fixes #8769.
2020-11-26 11:18:10 +00:00
Dmitry Borodaenko
7c43447477
Strip trailing / from server_url in register_new_matrix_user (#8823)
When server URL provided to register_new_matrix_user includes path
component (e.g. "http://localhost:8008/"), the command fails with
"ERROR! Received 400 Bad Request". Stripping trailing slash from the
server_url command argument makes sure combined endpoint URL remains
valid.

Signed-off-by: Dmitry Borodaenko angdraug@debian.org
2020-11-26 10:57:26 +00:00
Dirk Klimpel
14f81a6d24
Improve documentation how to configure prometheus for workers (#8822) 2020-11-26 10:42:55 +00:00
Dirk Klimpel
3f0ff53158
Remove deprecated /_matrix/client/*/admin endpoints (#8785)
These are now only available via `/_synapse/admin/v1`.
2020-11-25 16:26:11 -05:00
Andrew Morgan
2b110dda2a
Fix the formatting of push config section (#8818)
This PR updates the push config's formatting to better align with our [code style guidelines](https://github.com/matrix-org/synapse/blob/develop/docs/code_style.md#configuration-file-format).
2020-11-25 21:02:53 +00:00
Andrew Morgan
d963c69ba5
Speed up remote invite rejection database call (#8815)
This is another PR that grew out of #6739.

The existing code for checking whether a user is currently invited to a room when they want to leave the room looks like the following:

f737368a26/synapse/handlers/room_member.py (L518-L540)

It calls `get_invite_for_local_user_in_room`, which will actually query *all* rooms the user has been invited to, before iterating over them and matching via the room ID. It will then return a tuple of a lot of information which we pull the event ID out of.

I need to do a similar check for knocking, but this code wasn't very efficient. I then tried to write a different implementation using `StateHandler.get_current_state` but this actually didn't work as we haven't *joined* the room yet - we've only been invited to it. That means that only certain tables in Synapse have our desired `invite` membership state. One of those tables is `local_current_membership`.

So I wrote a store method that just queries that table instead
2020-11-25 20:06:13 +00:00
Patrick Cloke
968939bdac
Add additional type hints to HTTP client. (#8812)
This also removes some duplicated code between the simple
HTTP client and matrix federation client.
2020-11-25 13:30:47 -05:00
Patrick Cloke
4fd222ad70
Support trying multiple localparts for OpenID Connect. (#8801)
Abstracts the SAML and OpenID Connect code which attempts to regenerate
the localpart of a matrix ID if it is already in use.
2020-11-25 10:04:22 -05:00
Patrick Cloke
f38676d161
Add type hints to matrix federation client / agent. (#8806) 2020-11-25 07:07:21 -05:00
Dirk Klimpel
b08dc7effe
Clarify documentation of the admin list media API (#8795)
Clarify that the list media API only shows media from unencrypted events.
2020-11-24 09:04:51 -05:00
Richard van der Hoff
97b35ee259
Add a script to sign arbitrary json objects. (#8772) 2020-11-24 12:53:00 +00:00
Richard van der Hoff
e3d7806704
Update turn-howto (#8779)
Some hopefully-useful notes on setting up a turnserver.
2020-11-24 12:52:22 +00:00
Patrick Cloke
6fde6aa9c0
Properly report user-agent/IP during registration of SSO users. (#8784)
This also expands type-hints to the SSO and registration code.

Refactors the CAS code to more closely match OIDC/SAML.
2020-11-23 13:28:03 -05:00
Waylon Cude
7127855741
Fix synctl and duplicate worker spawning (#8798)
Synctl did not check if a worker thread was already running when using
`synctl start` and would naively start a fresh copy. This would
sometimes lead to cases where many duplicate copies of a single worker
would run.

This fix adds a pid check when starting worker threads and synctl will
now refuse to start individual workers if they're already running.
2020-11-23 15:20:49 +00:00
Andrew Morgan
59a995f38d
Improve logging of the mapping from SSO IDs to Matrix IDs. (#8773) 2020-11-23 08:45:23 -05:00
Daniele Sluijters
8ca120df7c
INSTALL: Fix setting content-type on well-known (#8793)
When using `add_header` nginx will literally add a header. If a
`content-type` header is already configured (for example through a
server wide default), this means we end up with 2 content-type headers,
like so:

```
content-type: text/html
content-type: application/json
access-control-allow-origin: *
```

That doesn't make sense. Instead, we want the content type of that
block to only be `application/json` which we can achieve using
`default_type` instead.

Signed-off-by: Daniele Sluijters <daenney@users.noreply.github.com>
2020-11-23 13:01:18 +00:00
Matthew Hodgson
476b8c0ae6 fix MD 2020-11-22 00:30:13 +00:00
Matthew Hodgson
1091bcea3e fix ancient changelog to be MD 2020-11-22 00:29:05 +00:00
Patrick Cloke
79bfe966e0
Improve error checking for OIDC/SAML mapping providers (#8774)
Checks that the localpart returned by mapping providers for SAML and
OIDC are valid before registering new users.

Extends the OIDC tests for existing users and invalid data.
2020-11-19 14:25:17 -05:00
Ben Banfield-Zanin
53a6f5ddf0
SAML: Allow specifying the IdP entityid to use. (#8630)
If the SAML metadata includes multiple IdPs it is necessary to
specify which IdP to redirect users to for authentication.
2020-11-19 09:57:13 -05:00
Richard van der Hoff
950bb0305f
Consistently use room_id from federation request body (#8776)
* Consistently use room_id from federation request body

Some federation APIs have a redundant `room_id` path param (see
https://github.com/matrix-org/matrix-doc/issues/2330). We should make sure we
consistently use either the path param or the body param, and the body param is
easier.

* Kill off some references to "context"

Once upon a time, "rooms" were known as "contexts". I think this kills of the
last references to "contexts".
2020-11-19 10:05:33 +00:00