Mathieu Velten
6cddf24e36
Faster joins: don't stall when a user joins during a fast join ( #14606 )
...
Fixes #12801 .
Complement tests are at
https://github.com/matrix-org/complement/pull/567 .
Avoid blocking on full state when handling a subsequent join into a
partial state room.
Also always perform a remote join into partial state rooms, since we do
not know whether the joining user has been banned and want to avoid
leaking history to banned users.
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <seanq@matrix.org>
Co-authored-by: David Robertson <davidr@element.io>
2023-02-10 23:31:05 +00:00
Patrick Cloke
cf5233b783
Avoid fetching unused account data in sync. ( #14973 )
...
The per-room account data is no longer unconditionally
fetched, even if all rooms will be filtered out.
Global account data will not be fetched if it will all be
filtered out.
2023-02-10 14:22:16 +00:00
Mathieu Velten
54c012c5a8
Make handle_new_client_event throws PartialStateConflictError ( #14665 )
...
Then adapts calling code to retry when needed so it doesn't 500
to clients.
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-12-15 16:04:23 +00:00
David Robertson
b5b5f66084
Move StateFilter to synapse.types ( #14668 )
...
* Move `StateFilter` to `synapse.types`
* Changelog
2022-12-12 16:19:30 +00:00
lukasdenk
719488dda8
Add query parameter ts to allow appservices set the origin_server_ts for state events. ( #11866 )
...
MSC3316 declares that both /rooms/{roomId}/send and /rooms/{roomId}/state
should accept a ts parameter for appservices. This change expands support
to /state and adds tests.
2022-10-03 13:30:45 +00:00
Eric Eastwood
a52c40e2a6
Fix get_users_in_room mis-use in transfer_room_state_on_room_upgrade ( #13960 )
...
Spawning from looking into `get_users_in_room` while investigating https://github.com/matrix-org/synapse/issues/13942#issuecomment-1262787050 .
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755 for the original exploration around finding `get_users_in_room` mis-uses.
Related to the following PRs where we also cleaned up some `get_users_in_room` mis-uses:
- https://github.com/matrix-org/synapse/pull/13605
- https://github.com/matrix-org/synapse/pull/13608
- https://github.com/matrix-org/synapse/pull/13606
- https://github.com/matrix-org/synapse/pull/13958
2022-09-30 20:10:50 -05:00
Shay
8ab16a92ed
Persist CreateRoom events to DB in a batch ( #13800 )
2022-09-28 10:11:48 +00:00
reivilibre
6302753012
Deduplicate is_server_notices_room. ( #13780 )
2022-09-14 15:53:18 +00:00
Jacek Kuśnierz
84ddcd7bbf
Drop support for calling /_matrix/client/v3/rooms/{roomId}/invite without an id_access_token ( #13241 )
...
Fixes #13206
Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de
2022-08-31 12:10:25 +00:00
Dirk Klimpel
682dfcfc0d
Fix that user cannot /forget rooms after the last member has left ( #13546 )
2022-08-30 09:58:38 +00:00
Eric Eastwood
d58615c82c
Directly lookup local membership instead of getting all members in a room first (get_users_in_room mis-use) ( #13608 )
...
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755
2022-08-24 14:13:12 -05:00
Dirk Klimpel
37f329c9ad
Fix that sending server notices fail if avatar is None ( #13566 )
...
Indroduced in #11846 .
2022-08-23 09:48:35 +01:00
Quentin Gliech
3dd175b628
synapse.api.auth.Auth cleanup: make permission-related methods use Requester instead of the UserID (#13024 )
...
Part of #13019
This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it.
It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.
2022-08-22 14:17:59 +01:00
Shay
a648a06d52
Add some tracing spans to give insight into local joins ( #13439 )
2022-08-03 10:19:34 -07:00
reivilibre
39be5bc550
Make minor clarifications to the error messages given when we fail to join a room via any server. ( #13160 )
2022-07-27 10:37:50 +00:00
David Robertson
b977867358
Rate limit joins per-room ( #13276 )
2022-07-19 11:45:17 +00:00
David Robertson
7281591f4c
Use state before join to determine if we _should_perform_remote_join ( #13270 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-07-15 12:20:47 +00:00
Nick Mills-Barrett
982fe29655
Optimise room creation event lookups part 2 ( #13224 )
2022-07-13 19:32:46 +01:00
David Teller
11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return Tuple[Codes, dict] ( #13044 )
...
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
Eric Eastwood
2c2a42cc10
Fix application service not being able to join remote federated room without a profile set ( #13131 )
...
Fix https://github.com/matrix-org/synapse/issues/4778
Complement tests: https://github.com/matrix-org/complement/pull/399
2022-07-05 05:56:06 -05:00
David Teller
80c7a06777
Rate limiting invites per issuer ( #13125 )
...
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-06-30 09:44:47 +00:00
reivilibre
7552615247
Reduce the duplication of code that invokes the rate limiter. ( #13070 )
2022-06-16 12:40:29 +01:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. ( #12857 )
...
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Erik Johnston
e3163e2e11
Reduce the amount of state we pull from the DB ( #12811 )
2022-06-06 09:24:12 +01:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state ( #12872 )
2022-06-01 16:02:53 +01:00
Patrick Cloke
a8db8c6eba
Remove user-visible groups/communities code ( #12553 )
...
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:
* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Shay
71e8afe34d
Update EventContext get_current_event_ids and get_prev_event_ids to accept state filters and update calls where possible ( #12791 )
2022-05-20 09:54:12 +01:00
Sean Quah
800ba87cc8
Refactor and convert Linearizer to async ( #12357 )
...
Refactor and convert `Linearizer` to async. This makes a `Linearizer`
cancellation bug easier to fix.
Also refactor to use an async context manager, which eliminates an
unlikely footgun where code that doesn't immediately use the context
manager could forget to release the lock.
Signed-off-by: Sean Quah <seanq@element.io>
2022-04-05 15:43:52 +01:00
Eric Eastwood
6f2943714b
Remove unused auth_event_ids argument plumbing ( #12304 )
...
Follow-up to https://github.com/matrix-org/synapse/pull/12083
Since we are now using the new `state_event_ids` parameter to do all of the heavy lifting.
We can remove any spots where we plumbed `auth_event_ids` just for MSC2716 things in
https://github.com/matrix-org/synapse/pull/9247/files .
Removing `auth_event_ids` from following functions:
- `create_and_send_nonmember_event`
- `_local_membership_update`
- `update_membership`
- `update_membership_locked`
2022-03-29 09:18:52 +01:00
Eric Eastwood
14662d3c18
Refactor create_new_client_event to use a new parameter, state_event_ids, which accurately describes the usage with MSC2716 instead of abusing auth_event_ids ( #12083 )
...
Spawned from https://github.com/matrix-org/synapse/pull/10975#discussion_r813183430
Part of [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716 )
2022-03-25 09:21:06 -05:00
Richard van der Hoff
8533c8b03d
Avoid generating state groups for local out-of-band leaves ( #12154 )
...
If we locally generate a rejection for an invite received over federation, it
is stored as an outlier (because we probably don't have the state for the
room). However, currently we still generate a state group for it (even though
the state in that state group will be nonsense).
By setting the `outlier` param on `create_event`, we avoid the nonsensical
state.
2022-03-03 19:58:08 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
Brendan Abolivier
130fd45393
Limit concurrent AS joins ( #11996 )
...
Initially introduced in matrix-org-hotfixes by e5537cfFixes #11995
See also #4826
2022-02-16 11:16:48 +00:00
Eric Eastwood
fef2e792be
Fix historical messages backfilling in random order on remote homeservers (MSC2716) ( #11114 )
...
Fix https://github.com/matrix-org/synapse/issues/11091
Fix https://github.com/matrix-org/synapse/issues/10764 (side-stepping the issue because we no longer have to deal with `fake_prev_event_id`)
1. Made the `/backfill` response return messages in `(depth, stream_ordering)` order (previously only sorted by `depth`)
- Technically, it shouldn't really matter how `/backfill` returns things but I'm just trying to make the `stream_ordering` a little more consistent from the origin to the remote homeservers in order to get the order of messages from `/messages` consistent ([sorted by `(topological_ordering, stream_ordering)`](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering )).
- Even now that we return backfilled messages in order, it still doesn't guarantee the same `stream_ordering` (and more importantly the [`/messages` order](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering )) on the other server. For example, if a room has a bunch of history imported and someone visits a permalink to a historical message back in time, their homeserver will skip over the historical messages in between and insert the permalink as the next message in the `stream_order` and totally throw off the sort.
- This will be even more the case when we add the [MSC3030 jump to date API endpoint](https://github.com/matrix-org/matrix-doc/pull/3030 ) so the static archives can navigate and jump to a certain date.
- We're solving this in the future by switching to [online topological ordering](https://github.com/matrix-org/gomatrixserverlib/issues/187 ) and [chunking](https://github.com/matrix-org/synapse/issues/3785 ) which by its nature will apply retroactively to fix any inconsistencies introduced by people permalinking
2. As we're navigating `prev_events` to return in `/backfill`, we order by `depth` first (newest -> oldest) and now also tie-break based on the `stream_ordering` (newest -> oldest). This is technically important because MSC2716 inserts a bunch of historical messages at the same `depth` so it's best to be prescriptive about which ones we should process first. In reality, I think the code already looped over the historical messages as expected because the database is already in order.
3. Making the historical state chain and historical event chain float on their own by having no `prev_events` instead of a fake `prev_event` which caused backfill to get clogged with an unresolvable event. Fixes https://github.com/matrix-org/synapse/issues/11091 and https://github.com/matrix-org/synapse/issues/10764
4. We no longer find connected insertion events by finding a potential `prev_event` connection to the current event we're iterating over. We now solely rely on marker events which when processed, add the insertion event as an extremity and the federating homeserver can ask about it when time calls.
- Related discussion, https://github.com/matrix-org/synapse/pull/11114#discussion_r741514793
Before | After
--- | ---
 | 
#### Why aren't we sorting topologically when receiving backfill events?
> The main reason we're going to opt to not sort topologically when receiving backfill events is because it's probably best to do whatever is easiest to make it just work. People will probably have opinions once they look at [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) which could change whatever implementation anyway.
>
> As mentioned, ideally we would do this but code necessary to make the fake edges but it gets confusing and gives an impression of “just whyyyy” (feels icky). This problem also dissolves with online topological ordering.
>
> -- https://github.com/matrix-org/synapse/pull/11114#discussion_r741517138
See https://github.com/matrix-org/synapse/pull/11114#discussion_r739610091 for the technical difficulties
2022-02-07 15:54:13 -06:00
Brendan Abolivier
d80d39b035
Add a ratelimiter for 3pid invite ( #11892 )
2022-02-03 14:28:15 +01:00
Brendan Abolivier
bf60da1a60
Configurable limits on avatars ( #11846 )
...
Only allow files which file size and content types match configured
limits to be set as avatar.
Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
2022-01-28 15:41:33 +01:00
Sean Quah
c7fe32edb4
Add type hints to synapse/storage/databases/main/room.py ( #11575 )
2021-12-15 18:00:48 +00:00
Eric Eastwood
aa8708ebed
Allow events to be created with no prev_events (MSC2716) ( #11243 )
...
The event still needs to have `auth_events` defined to be valid.
Split out from https://github.com/matrix-org/synapse/pull/11114
2021-12-10 23:08:51 -06:00
Eric Eastwood
7ffddd819c
Prevent historical state from being pushed to an application service via /transactions (MSC2716) ( #11265 )
...
Mark historical state from the MSC2716 `/batch_send` endpoint as `historical` which makes it `backfilled` and have a negative `stream_ordering` so it doesn't get queried by `/transactions`.
Fix https://github.com/matrix-org/synapse/issues/11241
Complement tests: https://github.com/matrix-org/complement/pull/221
2021-11-18 14:16:08 -06:00
Patrick Cloke
c01bc5f43d
Add remaining type hints to synapse.events. ( #11098 )
2021-11-02 09:55:52 -04:00
Patrick Cloke
eb9ddc8c2e
Remove the deprecated BaseHandler. ( #11005 )
...
The shared ratelimit function was replaced with a dedicated
RequestRatelimiter class (accessible from the HomeServer
object).
Other properties were copied to each sub-class that inherited
from BaseHandler.
2021-10-08 07:44:43 -04:00
Brendan Abolivier
4e51621064
Add a spamchecker method to allow or deny 3pid invites ( #10894 )
...
This is in the context of creating new module callbacks that modules in https://github.com/matrix-org/synapse-dinsic can use, in an effort to reconcile the spam checker API in synapse-dinsic with the one in mainline.
Note that a module callback already exists for 3pid invites (https://matrix-org.github.io/synapse/develop/modules/third_party_rules_callbacks.html#check_threepid_can_be_invited ) but it doesn't check whether the sender of the invite is allowed to send it.
2021-10-06 17:18:13 +02:00
Patrick Cloke
f4b1a9a527
Require direct references to configuration variables. ( #10985 )
...
This removes the magic allowing accessing configurable
variables directly from the config object. It is now required
that a specific configuration class is used (e.g. `config.foo`
must be replaced with `config.server.foo`).
2021-10-06 10:47:41 -04:00
Brendan Abolivier
829f2a82b0
Add a spamchecker callback to allow or deny room joins ( #10910 )
...
Co-authored-by: Erik Johnston <erik@matrix.org>
2021-10-06 14:32:16 +00:00
Patrick Cloke
a0f48ee89d
Use direct references for configuration variables (part 7). ( #10959 )
2021-10-04 07:18:54 -04:00
Patrick Cloke
d1bf5f7c9d
Strip "join_authorised_via_users_server" from join events which do not need it. ( #10933 )
...
This fixes a "Event not signed by authorising server" error when
transition room member from join -> join, e.g. when updating a
display name or avatar URL for restricted rooms.
2021-09-30 11:13:59 -04:00
Patrick Cloke
94b620a5ed
Use direct references for configuration variables (part 6). ( #10916 )
2021-09-29 06:44:15 -04:00
Patrick Cloke
bb7fdd821b
Use direct references for configuration variables (part 5). ( #10897 )
2021-09-24 07:25:21 -04:00
Aaron Raimist
6a751ff5e0
Allow sending a membership event to unban a user ( #10807 )
...
* Allow membership event to unban user
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2021-09-21 10:23:34 +00:00
Patrick Cloke
b3590614da
Require type hints in the handlers module. ( #10831 )
...
Adds missing type hints to methods in the synapse.handlers
module and requires all methods to have type hints there.
This also removes the unused construct_auth_difference method
from the FederationHandler.
2021-09-20 08:56:23 -04:00
