Commit Graph

367 Commits

Author SHA1 Message Date
Denis
c802ef1411
Don't include redundant prev_state in new events (#13791) 2022-09-20 09:44:38 +01:00
reivilibre
c2fe48a6ff
Rename the EventFormatVersions enum values so that they line up with room version numbers. (#13706) 2022-09-07 11:08:20 +01:00
Eric Eastwood
7af07f9716
Instrument _check_sigs_and_hash_and_fetch to trace time spent in child concurrent calls (#13588)
Instrument `_check_sigs_and_hash_and_fetch` to trace time spent in child concurrent calls because I've see `_check_sigs_and_hash_and_fetch` take [10.41s to process 100 events](https://github.com/matrix-org/synapse/issues/13587)

Fix https://github.com/matrix-org/synapse/issues/13587

Part of https://github.com/matrix-org/synapse/issues/13356
2022-08-23 21:53:37 -05:00
Eric Eastwood
06df5d4250
MSC2716v4 room version - remove namespace from MSC2716 event content fields (#13551)
Complement PR: https://github.com/matrix-org/complement/pull/450

As suggested in
https://github.com/matrix-org/matrix-spec-proposals/pull/2716#discussion_r941444525
2022-08-19 15:37:01 -05:00
Richard van der Hoff
96d92156d0
Update type of EventContext.rejected (#13460) 2022-08-04 17:45:01 +01:00
Erik Johnston
c6a05063ff
Don't pull out the full state when creating an event (#13281) 2022-07-18 10:05:30 +01:00
David Robertson
7281591f4c
Use state before join to determine if we _should_perform_remote_join (#13270)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-07-15 12:20:47 +00:00
David Teller
11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return Tuple[Codes, dict] (#13044)
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
David Teller
57f6f59e3e
Make _get_state_map_for_room not break when room state events don't contain an event id. (#13174)
Method `_get_state_map_for_room` seems to break in presence of some ill-formed events in the database. Reimplementing this method to use `get_current_state`, which is more robust to such events.
2022-07-07 08:14:32 +00:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. (#12857)
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Richard van der Hoff
68be42f6b6 Remove room_version param from validate_event_for_room_version
Instead, use the `room_version` property of the event we're validating.

The `room_version` was originally added as a parameter somewhere around #4482,
but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
2022-06-12 23:13:09 +01:00
Erik Johnston
888a29f412
Wait for lazy join to complete when getting current state (#12872) 2022-06-01 16:02:53 +01:00
Erik Johnston
3594f6c1f3 Merge branch 'master' into develop 2022-05-31 14:48:22 +01:00
Erik Johnston
1e453053cb
Rename storage classes (#12913) 2022-05-31 12:17:50 +00:00
Brendan Abolivier
8fd87739bf
Fix import in module_api module and docs on the new check_event_for_spam signature (#12918)
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-05-31 12:04:53 +02:00
David Teller
af7db19e1e
Uniformize spam-checker API, part 3: Expand check_event_for_spam with the ability to return additional fields (#12846)
Signed-off-by: David Teller <davidt@element.io>
2022-05-30 18:24:56 +02:00
David Robertson
796a0312e1
Bump jsonschema stubs (#12912) 2022-05-30 10:47:09 +01:00
David Teller
28199e9357
Uniformize spam-checker API, part 2: check_event_for_spam (#12808)
Signed-off-by: David Teller <davidt@element.io>
2022-05-23 17:27:39 +00:00
Jess Porter
a608ac847b
add SpamChecker callback for silently dropping inbound federated events (#12744)
Signed-off-by: jesopo <github@lolnerd.net>
2022-05-23 16:36:21 +00:00
Shay
71e8afe34d
Update EventContext get_current_event_ids and get_prev_event_ids to accept state filters and update calls where possible (#12791) 2022-05-20 09:54:12 +01:00
Patrick Cloke
86a515ccbf
Consolidate logic for parsing relations. (#12693)
Parse the `m.relates_to` event content field (which describes relations)
in a single place, this is used during:

* Event persistence.
* Validation of the Client-Server API.
* Fetching bundled aggregations.
* Processing of push rules.

Each of these separately implement the logic and each made slightly
different assumptions about what was valid. Some had minor / potential
bugs.
2022-05-16 12:42:45 +00:00
Jess Porter
39bed28b28
SpamChecker metrics (#12513)
* add Measure blocks all over SpamChecker

Signed-off-by: jesopo <github@lolnerd.net>

* fix test_spam_checker_may_join_room and test_threepid_invite_spamcheck

* better changelog entry
2022-05-13 12:17:38 +01:00
Erik Johnston
c72d26c1e1
Refactor EventContext (#12689)
Refactor how the `EventContext` class works, with the intention of reducing the amount of state we fetch from the DB during event processing.

The idea here is to get rid of the cached `current_state_ids` and `prev_state_ids` that live in the `EventContext`, and instead defer straight to the database (and its caching). 

One change that may have a noticeable effect is that we now no longer prefill the `get_current_state_ids` cache on a state change. However, that query is relatively light, since its just a case of reading a table from the DB (unlike fetching state at an event which is more heavyweight). For deployments with workers this cache isn't even used.


Part of #12684
2022-05-10 19:43:13 +00:00
Dirk Klimpel
989fa33096
Add some type hints to datastore. (#12477) 2022-05-10 14:07:48 -04:00
Sean Quah
a00462dd99
Implement cancellation support/protection for module callbacks (#12568)
There's no guarantee that module callbacks will handle cancellation
appropriately. Protect module callbacks with read semantics from
cancellation and avoid swallowing `CancelledError`s that arise.

Other module callbacks, such as the `on_*` callbacks, are presumed to
live on code paths that involve writes and aren't cancellation-friendly.
These module callbacks have been left alone.

Signed-off-by: Sean Quah <seanq@element.io>
2022-05-09 12:31:14 +01:00
David Robertson
fa0eab9c8e
Use ParamSpec in a few places (#12667) 2022-05-09 10:27:39 +00:00
David Robertson
051a1c3f22
Convert stringy power levels to integers on room upgrade (#12657) 2022-05-07 13:37:29 +01:00
David Robertson
2607b3e181
Update mypy to 0.950 and fix complaints (#12650) 2022-05-06 12:35:20 +00:00
Patrick Cloke
75dff3dc98
Include bundled aggregations for the latest event in a thread. (#12273)
The `latest_event` field of the bundled aggregations for `m.thread` relations
did not include bundled aggregations itself. This resulted in clients needing to
immediately request the event from the server (and thus making it useless that
the latest event itself was serialized instead of just including an event ID).
2022-05-04 08:38:18 -04:00
Richard van der Hoff
77dee1b451
fix imports
broken in 5938928 :-S
2022-05-03 13:59:28 +01:00
Richard van der Hoff
5938928c59
minor wording fix in docstring 2022-05-03 13:50:50 +01:00
Richard van der Hoff
db2edf5a65
Exclude OOB memberships from the federation sender (#12570)
As the comment says, there is no need to process such events, and indeed we
need to avoid doing so.

Fixes #12509.
2022-05-03 12:47:56 +00:00
Patrick Cloke
8a23bde823
Consistently use collections.abc.Mapping to check frozendict. (#12564) 2022-04-27 09:00:07 -04:00
Patrick Cloke
185da8f0f2
Misc. clean-ups to the relations code (#12519)
* Corrects some typos / copy & paste errors in tests.
* Clarifies docstrings.
* Removes an unnecessary method.
2022-04-25 08:25:56 -04:00
Richard van der Hoff
b80bb7e452
Fix /room/.../event/... to return the *original* event after any edits (#12476)
This is what the MSC (now) requires. Fixes https://github.com/matrix-org/synapse/issues/10310.
2022-04-19 16:42:19 +01:00
Patrick Cloke
86cf6a3a17
Remove references to unstable identifiers from MSC3440. (#12382)
Removes references to unstable thread relation, unstable
identifiers for filtering parameters, and the experimental
config flag.
2022-04-12 08:42:03 -04:00
Brendan Abolivier
5e88143dff
Add a callback to react to 3PID associations (#12302) 2022-03-31 18:27:21 +02:00
David Robertson
e0bb268134
Fix typechecker problems exposed by signedjson 1.1.2 (#12326) 2022-03-29 21:37:50 +00:00
David Robertson
a2b00a4486
Bump black and click versions (#12320) 2022-03-29 10:41:19 +00:00
Patrick Cloke
8fe930c215
Move get_bundled_aggregations to relations handler. (#12237)
The get_bundled_aggregations code is fairly high-level and uses
a lot of store methods, we move it into the handler as that seems
like a better fit.
2022-03-18 17:49:32 +00:00
David Robertson
872dbb0181
Correct check_username_for_spam annotations and docs (#12246)
* Formally type the UserProfile in user searches
* export UserProfile in synapse.module_api
* Update docs

Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
2022-03-18 13:51:41 +00:00
Patrick Cloke
96274565ff
Fix bundling aggregations if unsigned is not a returned event field. (#12234)
An error occured if a filter was supplied with `event_fields` which did not include
`unsigned`.

In that case, bundled aggregations are still added as the spec states it is allowed
for servers to add additional fields.
2022-03-16 12:17:39 -04:00
Shay
ed9aea42fa
fix misleading comment in check_events_for_spam (#12203) 2022-03-10 09:40:07 -08:00
Patrick Cloke
ea27528b5d
Support stable identifiers for MSC3440: Threading (#12151)
The unstable identifiers are still supported if the experimental configuration
flag is enabled. The unstable identifiers will be removed in a future release.
2022-03-10 15:36:13 +00:00
Will Hunt
15382b1afa
Add third_party module callbacks to check if a user can delete a room and deactivate a user (#12028)
* Add check_can_deactivate_user

* Add check_can_shutdown_rooms

* Documentation

* callbacks, not functions

* Various suggested tweaks

* Add tests for test_check_can_shutdown_room and test_check_can_deactivate_user

* Update check_can_deactivate_user to not take a Requester

* Fix check_can_shutdown_room docs

* Renegade and use `by_admin` instead of `admin_user_id`

* fix lint

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

* Update docs/modules/third_party_rules_callbacks.md

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>

Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-03-09 18:23:57 +00:00
Richard van der Hoff
00a67f831a Merge remote-tracking branch 'origin/release-v1.54' into develop 2022-03-04 22:40:51 +00:00
Patrick Cloke
1d11b452b7
Use the proper serialization format when bundling aggregations. (#12090)
This ensures that the `latest_event` field of the bundled aggregation
for threads uses the same format as the other events in the response.
2022-03-03 10:43:06 -05:00
Brendan Abolivier
ae8a616b49
Correctly register deactivation and profile update module callbacks (#12141) 2022-03-03 11:39:58 +01:00
Brendan Abolivier
300ed0b8a6
Add module callbacks called for reacting to deactivation status change and profile update (#12062) 2022-03-01 15:00:03 +00:00
Richard van der Hoff
e2e1d90a5e
Faster joins: persist to database (#12012)
When we get a partial_state response from send_join, store information in the
database about it:
 * store a record about the room as a whole having partial state, and stash the
   list of member servers too.
 * flag the join event itself as having partial state
 * also, for any new events whose prev-events are partial-stated, note that
   they will *also* be partial-stated.

We don't yet make any attempt to interpret this data, so API calls (and a bunch
of other things) are just going to get incorrect data.
2022-03-01 12:49:54 +00:00