Mathieu Velten
8c3bcea2da
Rename pagination&purge locks and add comments explaining them ( #16112 )
2023-08-16 16:19:54 +02:00
Patrick Cloke
b7695ac388
Combine duplicated code for calculating an event ID from a txn ID ( #16023 )
...
Refactoring related to stabilization of MSC3970, refactor to combine
code which has the same logic.
2023-07-31 08:44:45 -04:00
Erik Johnston
ae55cc1e6b
Add ability to wait for locks and add locks to purge history / room deletion ( #15791 )
...
c.f. #13476
2023-07-31 10:58:03 +01:00
Erik Johnston
ad52db3b5c
Reduce the amount of state we pull out ( #15968 )
2023-07-20 10:46:37 +01:00
Eric Eastwood
0f02f0b4da
Remove experimental MSC2716 implementation to incrementally import history into existing rooms ( #15748 )
...
Context for why we're removing the implementation:
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
- https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734
Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737
Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
Tulir Asokan
1404f68a03
Fix joining rooms through aliases where the alias server isn't a real homeserver ( #15776 )
2023-06-14 15:42:33 +01:00
Andrew Morgan
2e59e97ebd
Move ThirdPartyEventRules into module_api/callbacks ( #15535 )
2023-05-04 14:18:22 +00:00
Sean Quah
04e79e6a18
Add config option to forget rooms automatically when users leave them ( #15224 )
...
This is largely based off the stats and user directory updater code.
Signed-off-by: Sean Quah <seanq@matrix.org>
2023-05-03 12:27:33 +01:00
Quentin Gliech
8b3a502996
Experimental support for MSC3970: per-device transaction IDs ( #15318 )
2023-04-25 09:37:09 +01:00
Andrew Morgan
aec639e3e3
Move Spam Checker callbacks to a dedicated file ( #15453 )
2023-04-18 00:57:40 +00:00
DeepBlueV7.X
753d1d9cde
Fix joining rooms you have been unbanned from ( #15323 )
...
* Fix joining rooms you have been unbanned from
Since forever synapse did not allow you to join a room after you have
been unbanned from it over federation. This was not actually because of
the unban event not federating. Synapse simply used outdated state to
validate the join transition. This skips the validation if we are not in
the room and for that reason won't have the current room state.
Fixes #1563
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* Add changelog
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
* Update changelog.d/15323.bugfix
---------
Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
2023-03-29 08:37:27 +00:00
Dirk Klimpel
ecbe0ddbe7
Add support for knocking to workers. ( #15133 )
2023-03-02 12:59:53 -05:00
Shay
1c95ddd09b
Batch up storing state groups when creating new room ( #14918 )
2023-02-24 13:15:29 -08:00
Mathieu Velten
6cddf24e36
Faster joins: don't stall when a user joins during a fast join ( #14606 )
...
Fixes #12801 .
Complement tests are at
https://github.com/matrix-org/complement/pull/567 .
Avoid blocking on full state when handling a subsequent join into a
partial state room.
Also always perform a remote join into partial state rooms, since we do
not know whether the joining user has been banned and want to avoid
leaking history to banned users.
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <seanq@matrix.org>
Co-authored-by: David Robertson <davidr@element.io>
2023-02-10 23:31:05 +00:00
Patrick Cloke
cf5233b783
Avoid fetching unused account data in sync. ( #14973 )
...
The per-room account data is no longer unconditionally
fetched, even if all rooms will be filtered out.
Global account data will not be fetched if it will all be
filtered out.
2023-02-10 14:22:16 +00:00
Mathieu Velten
54c012c5a8
Make handle_new_client_event throws PartialStateConflictError ( #14665 )
...
Then adapts calling code to retry when needed so it doesn't 500
to clients.
Signed-off-by: Mathieu Velten <mathieuv@matrix.org>
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-12-15 16:04:23 +00:00
David Robertson
b5b5f66084
Move StateFilter to synapse.types ( #14668 )
...
* Move `StateFilter` to `synapse.types`
* Changelog
2022-12-12 16:19:30 +00:00
lukasdenk
719488dda8
Add query parameter ts to allow appservices set the origin_server_ts for state events. ( #11866 )
...
MSC3316 declares that both /rooms/{roomId}/send and /rooms/{roomId}/state
should accept a ts parameter for appservices. This change expands support
to /state and adds tests.
2022-10-03 13:30:45 +00:00
Eric Eastwood
a52c40e2a6
Fix get_users_in_room mis-use in transfer_room_state_on_room_upgrade ( #13960 )
...
Spawning from looking into `get_users_in_room` while investigating https://github.com/matrix-org/synapse/issues/13942#issuecomment-1262787050 .
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755 for the original exploration around finding `get_users_in_room` mis-uses.
Related to the following PRs where we also cleaned up some `get_users_in_room` mis-uses:
- https://github.com/matrix-org/synapse/pull/13605
- https://github.com/matrix-org/synapse/pull/13608
- https://github.com/matrix-org/synapse/pull/13606
- https://github.com/matrix-org/synapse/pull/13958
2022-09-30 20:10:50 -05:00
Shay
8ab16a92ed
Persist CreateRoom events to DB in a batch ( #13800 )
2022-09-28 10:11:48 +00:00
reivilibre
6302753012
Deduplicate is_server_notices_room. ( #13780 )
2022-09-14 15:53:18 +00:00
Jacek Kuśnierz
84ddcd7bbf
Drop support for calling /_matrix/client/v3/rooms/{roomId}/invite without an id_access_token ( #13241 )
...
Fixes #13206
Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de
2022-08-31 12:10:25 +00:00
Dirk Klimpel
682dfcfc0d
Fix that user cannot /forget rooms after the last member has left ( #13546 )
2022-08-30 09:58:38 +00:00
Eric Eastwood
d58615c82c
Directly lookup local membership instead of getting all members in a room first (get_users_in_room mis-use) ( #13608 )
...
See https://github.com/matrix-org/synapse/pull/13575#discussion_r953023755
2022-08-24 14:13:12 -05:00
Dirk Klimpel
37f329c9ad
Fix that sending server notices fail if avatar is None ( #13566 )
...
Indroduced in #11846 .
2022-08-23 09:48:35 +01:00
Quentin Gliech
3dd175b628
synapse.api.auth.Auth cleanup: make permission-related methods use Requester instead of the UserID (#13024 )
...
Part of #13019
This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it.
It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.
2022-08-22 14:17:59 +01:00
Shay
a648a06d52
Add some tracing spans to give insight into local joins ( #13439 )
2022-08-03 10:19:34 -07:00
reivilibre
39be5bc550
Make minor clarifications to the error messages given when we fail to join a room via any server. ( #13160 )
2022-07-27 10:37:50 +00:00
David Robertson
b977867358
Rate limit joins per-room ( #13276 )
2022-07-19 11:45:17 +00:00
David Robertson
7281591f4c
Use state before join to determine if we _should_perform_remote_join ( #13270 )
...
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-07-15 12:20:47 +00:00
Nick Mills-Barrett
982fe29655
Optimise room creation event lookups part 2 ( #13224 )
2022-07-13 19:32:46 +01:00
David Teller
11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return Tuple[Codes, dict] ( #13044 )
...
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
Eric Eastwood
2c2a42cc10
Fix application service not being able to join remote federated room without a profile set ( #13131 )
...
Fix https://github.com/matrix-org/synapse/issues/4778
Complement tests: https://github.com/matrix-org/complement/pull/399
2022-07-05 05:56:06 -05:00
David Teller
80c7a06777
Rate limiting invites per issuer ( #13125 )
...
Co-authored-by: reivilibre <oliverw@matrix.org>
2022-06-30 09:44:47 +00:00
reivilibre
7552615247
Reduce the duplication of code that invokes the rate limiter. ( #13070 )
2022-06-16 12:40:29 +01:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. ( #12857 )
...
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Erik Johnston
e3163e2e11
Reduce the amount of state we pull from the DB ( #12811 )
2022-06-06 09:24:12 +01:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state ( #12872 )
2022-06-01 16:02:53 +01:00
Patrick Cloke
a8db8c6eba
Remove user-visible groups/communities code ( #12553 )
...
Makes it so that groups/communities no longer exist from a user-POV. E.g. we remove:
* All API endpoints (including Client-Server, Server-Server, and admin).
* Documented configuration options (and the experimental flag, which is now unused).
* Special handling during room upgrades.
* The `groups` section of the `/sync` response.
2022-05-25 07:53:40 -04:00
Shay
71e8afe34d
Update EventContext get_current_event_ids and get_prev_event_ids to accept state filters and update calls where possible ( #12791 )
2022-05-20 09:54:12 +01:00
Sean Quah
800ba87cc8
Refactor and convert Linearizer to async ( #12357 )
...
Refactor and convert `Linearizer` to async. This makes a `Linearizer`
cancellation bug easier to fix.
Also refactor to use an async context manager, which eliminates an
unlikely footgun where code that doesn't immediately use the context
manager could forget to release the lock.
Signed-off-by: Sean Quah <seanq@element.io>
2022-04-05 15:43:52 +01:00
Eric Eastwood
6f2943714b
Remove unused auth_event_ids argument plumbing ( #12304 )
...
Follow-up to https://github.com/matrix-org/synapse/pull/12083
Since we are now using the new `state_event_ids` parameter to do all of the heavy lifting.
We can remove any spots where we plumbed `auth_event_ids` just for MSC2716 things in
https://github.com/matrix-org/synapse/pull/9247/files .
Removing `auth_event_ids` from following functions:
- `create_and_send_nonmember_event`
- `_local_membership_update`
- `update_membership`
- `update_membership_locked`
2022-03-29 09:18:52 +01:00
Eric Eastwood
14662d3c18
Refactor create_new_client_event to use a new parameter, state_event_ids, which accurately describes the usage with MSC2716 instead of abusing auth_event_ids ( #12083 )
...
Spawned from https://github.com/matrix-org/synapse/pull/10975#discussion_r813183430
Part of [MSC2716](https://github.com/matrix-org/matrix-spec-proposals/pull/2716 )
2022-03-25 09:21:06 -05:00
Richard van der Hoff
8533c8b03d
Avoid generating state groups for local out-of-band leaves ( #12154 )
...
If we locally generate a rejection for an invite received over federation, it
is stored as an outlier (because we probably don't have the state for the
room). However, currently we still generate a state group for it (even though
the state in that state group will be nonsense).
By setting the `outlier` param on `create_event`, we avoid the nonsensical
state.
2022-03-03 19:58:08 +00:00
Richard van der Hoff
e24ff8ebe3
Remove HomeServer.get_datastore() ( #12031 )
...
The presence of this method was confusing, and mostly present for backwards
compatibility. Let's get rid of it.
Part of #11733
2022-02-23 11:04:02 +00:00
Brendan Abolivier
130fd45393
Limit concurrent AS joins ( #11996 )
...
Initially introduced in matrix-org-hotfixes by e5537cfFixes #11995
See also #4826
2022-02-16 11:16:48 +00:00
Eric Eastwood
fef2e792be
Fix historical messages backfilling in random order on remote homeservers (MSC2716) ( #11114 )
...
Fix https://github.com/matrix-org/synapse/issues/11091
Fix https://github.com/matrix-org/synapse/issues/10764 (side-stepping the issue because we no longer have to deal with `fake_prev_event_id`)
1. Made the `/backfill` response return messages in `(depth, stream_ordering)` order (previously only sorted by `depth`)
- Technically, it shouldn't really matter how `/backfill` returns things but I'm just trying to make the `stream_ordering` a little more consistent from the origin to the remote homeservers in order to get the order of messages from `/messages` consistent ([sorted by `(topological_ordering, stream_ordering)`](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering )).
- Even now that we return backfilled messages in order, it still doesn't guarantee the same `stream_ordering` (and more importantly the [`/messages` order](https://github.com/matrix-org/synapse/blob/develop/docs/development/room-dag-concepts.md#depth-and-stream-ordering )) on the other server. For example, if a room has a bunch of history imported and someone visits a permalink to a historical message back in time, their homeserver will skip over the historical messages in between and insert the permalink as the next message in the `stream_order` and totally throw off the sort.
- This will be even more the case when we add the [MSC3030 jump to date API endpoint](https://github.com/matrix-org/matrix-doc/pull/3030 ) so the static archives can navigate and jump to a certain date.
- We're solving this in the future by switching to [online topological ordering](https://github.com/matrix-org/gomatrixserverlib/issues/187 ) and [chunking](https://github.com/matrix-org/synapse/issues/3785 ) which by its nature will apply retroactively to fix any inconsistencies introduced by people permalinking
2. As we're navigating `prev_events` to return in `/backfill`, we order by `depth` first (newest -> oldest) and now also tie-break based on the `stream_ordering` (newest -> oldest). This is technically important because MSC2716 inserts a bunch of historical messages at the same `depth` so it's best to be prescriptive about which ones we should process first. In reality, I think the code already looped over the historical messages as expected because the database is already in order.
3. Making the historical state chain and historical event chain float on their own by having no `prev_events` instead of a fake `prev_event` which caused backfill to get clogged with an unresolvable event. Fixes https://github.com/matrix-org/synapse/issues/11091 and https://github.com/matrix-org/synapse/issues/10764
4. We no longer find connected insertion events by finding a potential `prev_event` connection to the current event we're iterating over. We now solely rely on marker events which when processed, add the insertion event as an extremity and the federating homeserver can ask about it when time calls.
- Related discussion, https://github.com/matrix-org/synapse/pull/11114#discussion_r741514793
Before | After
--- | ---
 | 
#### Why aren't we sorting topologically when receiving backfill events?
> The main reason we're going to opt to not sort topologically when receiving backfill events is because it's probably best to do whatever is easiest to make it just work. People will probably have opinions once they look at [MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716 ) which could change whatever implementation anyway.
>
> As mentioned, ideally we would do this but code necessary to make the fake edges but it gets confusing and gives an impression of “just whyyyy” (feels icky). This problem also dissolves with online topological ordering.
>
> -- https://github.com/matrix-org/synapse/pull/11114#discussion_r741517138
See https://github.com/matrix-org/synapse/pull/11114#discussion_r739610091 for the technical difficulties
2022-02-07 15:54:13 -06:00
Brendan Abolivier
d80d39b035
Add a ratelimiter for 3pid invite ( #11892 )
2022-02-03 14:28:15 +01:00
Brendan Abolivier
bf60da1a60
Configurable limits on avatars ( #11846 )
...
Only allow files which file size and content types match configured
limits to be set as avatar.
Most of the inspiration from the non-test code comes from matrix-org/synapse-dinsic#19
2022-01-28 15:41:33 +01:00
Sean Quah
c7fe32edb4
Add type hints to synapse/storage/databases/main/room.py ( #11575 )
2021-12-15 18:00:48 +00:00
