Shay
a648a06d52
Add some tracing spans to give insight into local joins ( #13439 )
2022-08-03 10:19:34 -07:00
reivilibre
39be5bc550
Make minor clarifications to the error messages given when we fail to join a room via any server. ( #13160 )
2022-07-27 10:37:50 +00:00
David Robertson
b977867358
Rate limit joins per-room ( #13276 )
2022-07-19 11:45:17 +00:00
David Robertson
7281591f4c
Use state before join to determine if we _should_perform_remote_join ( #13270 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-07-15 12:20:47 +00:00
Nick Mills-Barrett
982fe29655
Optimise room creation event lookups part 2 ( #13224 )
2022-07-13 19:32:46 +01:00
David Teller
11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return Tuple[Codes, dict] ( #13044 )
...
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
Eric Eastwood
2c2a42cc10
Fix application service not being able to join remote federated room without a profile set ( #13131 )
...
Fix https://github.com/matrix-org/synapse/issues/4778
Complement tests: https://github.com/matrix-org/complement/pull/399
2022-07-05 05:56:06 -05:00
David Teller
80c7a06777
Rate limiting invites per issuer ( #13125 )
...
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-06-30 09:44:47 +00:00
reivilibre
7552615247
Reduce the duplication of code that invokes the rate limiter. ( #13070 )
2022-06-16 12:40:29 +01:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. ( #12857 )
...
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Erik Johnston
e3163e2e11
Reduce the amount of state we pull from the DB ( #12811 )
2022-06-06 09:24:12 +01:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state ( #12872 )
2022-06-01 16:02:53 +01:00
Patrick Cloke
a8db8c6eba
Remove user-visible groups/communities code ( #12553 )
...
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:
* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Shay
71e8afe34d
Update EventContext get_current_event_ids and get_prev_event_ids to accept state filters and update calls where possible ( #12791 )
2022-05-20 09:54:12 +01:00
Sean Quah
800ba87cc8
Refactor and convert Linearizer to async ( #12357 )
...
Refactor and convert `Linearizer` to async. This makes a `Linearizer`
cancellation bug easier to fix.
Also refactor to use an async context manager, which eliminates an
unlikely footgun where code that doesn't immediately use the context
manager could forget to release the lock.
Signed-off-by: Sean Quah <seanq@element.io>
2022-04-05 15:43:52 +01:00
Eric Eastwood
6f2943714b
Remove unused auth_event_ids argument plumbing ( #12304 )
...
Follow-up to https://github.com/matrix-org/synapse/pull/12083
Since we are now using the new `state_event_ids` parameter to do all of the heavy lifting.
We can remove any spots where we plumbed `auth_event_ids` just for MSC2716 things in
https://github.com/matrix-org/synapse/pull/9247/files .
Removing `auth_event_ids` from following functions:
- `create_and_send_nonmember_event`
- `_local_membership_update`
- `update_membership`
- `update_membership_locked`
2022-03-29 09:18:52 +01:00
Eric Eastwood
14662d3c18
Refactor create_new_client_event to use a new parameter, state_event_ids, which accurately describes the usage with MSC2716 instead of abusing auth_event_ids ( #12083 )
...
Spawned from https://github.com/matrix-org/synapse/pull/10975#discussion_r813183430
Part of [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716 )
2022-03-25 09:21:06 -05:00
Richard van der Hoff
8533c8b03d
Avoid generating state groups for local out-of-band leaves ( #12154 )
...
If we locally generate a rejection for an invite received over federation, it
is stored as an outlier (because we probably don't have the state for the
room). However, currently we still generate a state group for it (even though
the state in that state group will be nonsense).
By setting the `outlier` param on `create_event`, we avoid the nonsensical
state.
2022-03-03 19:58:08 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
Brendan Abolivier
130fd45393
Limit concurrent AS joins ( #11996 )
...
Initially introduced in matrix-org-hotfixes by e5537cfFixes #11995
See also #4826
2022-02-16 11:16:48 +00:00
Eric Eastwood
fef2e792be
Fix historical messages backfilling in random order on remote homeservers (MSC2716) ( #11114 )
...
Fix https://github.com/matrix-org/synapse/issues/11091
Fix https://github.com/matrix-org/synapse/issues/10764 (side-stepping the issue because we no longer have to deal with `fake_prev_event_id`)
1. Made the `/backfill` response return messages in `(depth, stream_ordering)` order (previously only sorted by `depth`)
- Technically, it shouldn't really matter how `/backfill` returns things but I'm just trying to make the `stream_ordering` a little more consistent from the origin to the remote homeservers in order to get the order of messages from `/messages` consistent ([sorted by `(topological_ordering, stream_ordering)`](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering )).
- Even now that we return backfilled messages in order, it still doesn't guarantee the same `stream_ordering` (and more importantly the [`/messages` order](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering )) on the other server. For example, if a room has a bunch of history imported and someone visits a permalink to a historical message back in time, their homeserver will skip over the historical messages in between and insert the permalink as the next message in the `stream_order` and totally throw off the sort.
- This will be even more the case when we add the [MSC3030 jump to date API endpoint](https://github.com/matrix-org/matrix-doc/pull/3030 ) so the static archives can navigate and jump to a certain date.
- We're solving this in the future by switching to [online topological ordering](https://github.com/matrix-org/gomatrixserverlib/issues/187 ) and [chunking](https://github.com/matrix-org/synapse/issues/3785 ) which by its nature will apply retroactively to fix any inconsistencies introduced by people permalinking
2. As we're navigating `prev_events` to return in `/backfill`, we order by `depth` first (newest -> oldest) and now also tie-break based on the `stream_ordering` (newest -> oldest). This is technically important because MSC2716 inserts a bunch of historical messages at the same `depth` so it's best to be prescriptive about which ones we should process first. In reality, I think the code already looped over the historical messages as expected because the database is already in order.
3. Making the historical state chain and historical event chain float on their own by having no `prev_events` instead of a fake `prev_event` which caused backfill to get clogged with an unresolvable event. Fixes https://github.com/matrix-org/synapse/issues/11091 and https://github.com/matrix-org/synapse/issues/10764
4. We no longer find connected insertion events by finding a potential `prev_event` connection to the current event we're iterating over. We now solely rely on marker events which when processed, add the insertion event as an extremity and the federating homeserver can ask about it when time calls.
- Related discussion, https://github.com/matrix-org/synapse/pull/11114#discussion_r741514793
Before | After
--- | ---
 | 
#### Why aren't we sorting topologically when receiving backfill events?
> The main reason we're going to opt to not sort topologically when receiving backfill events is because it's probably best to do whatever is easiest to make it just work. People will probably have opinions once they look at [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) which could change whatever implementation anyway.
>
> As mentioned, ideally we would do this but code necessary to make the fake edges but it gets confusing and gives an impression of “just whyyyy” (feels icky). This problem also dissolves with online topological ordering.
>
> -- https://github.com/matrix-org/synapse/pull/11114#discussion_r741517138
See https://github.com/matrix-org/synapse/pull/11114#discussion_r739610091 for the technical difficulties
2022-02-07 15:54:13 -06:00
Brendan Abolivier
d80d39b035
Add a ratelimiter for 3pid invite ( #11892 )
2022-02-03 14:28:15 +01:00
Brendan Abolivier
bf60da1a60
Configurable limits on avatars ( #11846 )
...
Only allow files which file size and content types match configured
limits to be set as avatar.
Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
2022-01-28 15:41:33 +01:00
Sean Quah
c7fe32edb4
Add type hints to synapse/storage/databases/main/room.py ( #11575 )
2021-12-15 18:00:48 +00:00
Eric Eastwood
aa8708ebed
Allow events to be created with no prev_events (MSC2716) ( #11243 )
...
The event still needs to have `auth_events` defined to be valid.
Split out from https://github.com/matrix-org/synapse/pull/11114
2021-12-10 23:08:51 -06:00
Eric Eastwood
7ffddd819c
Prevent historical state from being pushed to an application service via /transactions (MSC2716) ( #11265 )
...
Mark historical state from the MSC2716 `/batch_send` endpoint as `historical` which makes it `backfilled` and have a negative `stream_ordering` so it doesn't get queried by `/transactions`.
Fix https://github.com/matrix-org/synapse/issues/11241
Complement tests: https://github.com/matrix-org/complement/pull/221
2021-11-18 14:16:08 -06:00
Patrick Cloke
c01bc5f43d
Add remaining type hints to synapse.events. ( #11098 )
2021-11-02 09:55:52 -04:00
Patrick Cloke
eb9ddc8c2e
Remove the deprecated BaseHandler. ( #11005 )
...
The shared ratelimit function was replaced with a dedicated
RequestRatelimiter class (accessible from the HomeServer
object).
Other properties were copied to each sub-class that inherited
from BaseHandler.
2021-10-08 07:44:43 -04:00
Brendan Abolivier
4e51621064
Add a spamchecker method to allow or deny 3pid invites ( #10894 )
...
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.
Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited ) but it doesn't check whether the sender of the invite is allowed to send it.
2021-10-06 17:18:13 +02:00
Patrick Cloke
f4b1a9a527
Require direct references to configuration variables. ( #10985 )
...
This removes the magic allowing accessing configurable
variables directly from the config object. It is now required
that a specific configuration class is used (e.g. `config.foo`
must be replaced with `config.server.foo`).
2021-10-06 10:47:41 -04:00
Brendan Abolivier
829f2a82b0
Add a spamchecker callback to allow or deny room joins ( #10910 )
...
Co-authored-by: Erik Johnston <erik@matrix.org>
2021-10-06 14:32:16 +00:00
Patrick Cloke
a0f48ee89d
Use direct references for configuration variables (part 7). ( #10959 )
2021-10-04 07:18:54 -04:00
Patrick Cloke
d1bf5f7c9d
Strip "join_authorised_via_users_server" from join events which do not need it. ( #10933 )
...
This fixes a "Event not signed by authorising server" error when
transition room member from join -> join, e.g. when updating a
display name or avatar URL for restricted rooms.
2021-09-30 11:13:59 -04:00
Patrick Cloke
94b620a5ed
Use direct references for configuration variables (part 6). ( #10916 )
2021-09-29 06:44:15 -04:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). ( #10897 )
2021-09-24 07:25:21 -04:00
Aaron Raimist
6a751ff5e0
Allow sending a membership event to unban a user ( #10807 )
...
* Allow membership event to unban user
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-09-21 10:23:34 +00:00
Patrick Cloke
b3590614da
Require type hints in the handlers module. ( #10831 )
...
Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.
This also removes the unused construct_auth_difference method
from the FederationHandler.
2021-09-20 08:56:23 -04:00
Richard van der Hoff
56e2a30634
Move maybe_kick_guest_users out of BaseHandler ( #10744 )
...
This is part of my ongoing war against BaseHandler. I've moved kick_guest_users into RoomMemberHandler (since it calls out to that handler anyway), and split maybe_kick_guest_users into the two places it is called.
2021-09-06 12:17:16 +01:00
Azrenbeth
0c1d6f65d7
Enforce the max length for per-room display names / avatar URLs. ( #10654 )
...
To match the maximum lengths allowed for profile data.
2021-08-23 11:25:33 -04:00
Michael Telatynski
9db24cc50d
Send unstable-prefixed room_type in store-invite IS API requests ( #10435 )
...
The room type is per MSC3288 to allow the identity-server to
change invitation wording based on whether the invitation is to
a room or a space.
The prefixed key will be replaced once MSC3288 is accepted
into the spec.
2021-08-04 13:39:57 -04:00
Patrick Cloke
228decfce1
Update the MSC3083 support to verify if joins are from an authorized server. ( #10254 )
2021-07-26 12:17:00 -04:00
Eric Eastwood
96f6293de5
Add endpoints for backfilling history (MSC2716) ( #9247 )
...
Work on https://github.com/matrix-org/matrix-doc/pull/2716
2021-06-22 10:02:53 +01:00
Patrick Cloke
9e5ab6dd58
Remove the experimental flag for knocking and use stable prefixes / endpoints. ( #10167 )
...
* Room version 7 for knocking.
* Stable prefixes and endpoints (both client and federation) for knocking.
* Removes the experimental configuration flag.
2021-06-15 07:45:14 -04:00
Sorunome
d936371b69
Implement knock feature ( #6739 )
...
This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403
Signed-off-by: Sorunome mail@sorunome.de
Signed-off-by: Andrew Morgan andrewm@element.io
2021-06-09 19:39:51 +01:00
Patrick Cloke
ac6bfcd52f
Refactor checking restricted join rules ( #10007 )
...
To be more consistent with similar code. The check now automatically
raises an AuthError instead of passing back a boolean. It also absorbs
some shared logic between callers.
2021-05-18 12:17:04 -04:00
Brendan Abolivier
a683028d81
Correctly ratelimit invites when creating a room ( #9968 )
...
* Correctly ratelimit invites when creating a room
Also allow ratelimiting for more than one action at a time.
2021-05-12 16:05:28 +02:00
Patrick Cloke
bb4b11846f
Add missing type hints to handlers and fix a Spam Checker type hint. ( #9896 )
...
The user_may_create_room_alias method on spam checkers
declared the room_alias parameter as a str when in reality it is
passed a RoomAlias object.
2021-04-29 07:17:28 -04:00
Patrick Cloke
d924827da1
Check for space membership during a remote join of a restricted room ( #9814 )
...
When receiving a /send_join request for a room with join rules set to 'restricted',
check if the user is a member of the spaces defined in the 'allow' key of the join rules.
This only applies to an experimental room version, as defined in MSC3083.
2021-04-23 07:05:51 -04:00
Patrick Cloke
e8816c6ace
Revert "Check for space membership during a remote join of a restricted room. ( #9763 )"
...
This reverts commit cc51aaaa7a
2021-04-14 12:33:37 -04:00
Patrick Cloke
cc51aaaa7a
Check for space membership during a remote join of a restricted room. ( #9763 )
...
When receiving a /send_join request for a room with join rules set to 'restricted',
check if the user is a member of the spaces defined in the 'allow' key of the join
rules.
This only applies to an experimental room version, as defined in MSC3083.
2021-04-14 12:32:20 -04:00
