Commit Graph

91 Commits

Author SHA1 Message Date
Eric Eastwood
0f02f0b4da
Remove experimental MSC2716 implementation to incrementally import history into existing rooms (#15748)
Context for why we're removing the implementation:

 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1487441010
 - https://github.com/matrix-org/matrix-spec-proposals/pull/2716#issuecomment-1504262734

Anyone wanting to continue MSC2716, should also address these leftover tasks: https://github.com/matrix-org/synapse/issues/10737

Closes https://github.com/matrix-org/synapse/issues/10737 in the fact that it is not longer necessary to track those things.
2023-06-16 14:12:24 -05:00
Patrick Cloke
eb3c1823d8
Reject instead of erroring on invalid membership events. (#15564)
Instead of resulting in an internal server error for invalid events,
return that the event is invalid.
2023-05-15 15:01:29 -04:00
Patrick Cloke
2503126d52
Implement MSC2174: move redacts to a content property. (#15395)
This moves `redacts` from being a top-level property to
a `content` property in a new room version.

MSC2176 (which was previously implemented) states to not
`redact` this property.
2023-04-13 13:47:07 +00:00
Patrick Cloke
d07d255830
Implement MSC2175: remove the creator field from create events. (#15394) 2023-04-06 16:26:28 -04:00
Shay
a368d30c1c
More speedups/fixes to creating batched events (#15195) 2023-03-07 13:54:39 -08:00
Harishankar Kumar
db2b105d69
Change collection[str] to StrCollection in event_auth code (#14929)
Signed-off-by: Harishankar Kumar <hari01584@gmail.com>
2023-02-14 09:37:08 +00:00
David Robertson
a134e626e4
Reject boolean power levels (#14944)
* Better test for bad values in power levels events

The previous test only checked that Synapse didn't raise an exception,
but didn't check that we had correctly interpreted the value of the
dodgy power level.

It also conflated two things: bad room notification levels, and bad user
levels. There _is_ logic for converting the latter to integers, but we
should test it separately.

* Check we ignore types that don't convert to int

* Handle `None` values in `notifications.room`

* Changelog

* Also test that bad values are rejected by event auth

* Docstring

* linter scripttttttttt

* Test boolean values in PL content

* Reject boolean power levels

* Changelog
2023-01-31 10:57:02 +00:00
Patrick Cloke
4d6b1d3c47
Properly check for frozendicts in event auth code. (#14864)
Check for for an instance of a mapping instead of a dict.

This only affects room version 10 when frozen events are enabled.
2023-01-18 09:27:57 -05:00
reivilibre
62ed877433
Improve validation of field size limits in events. (#14664) 2022-12-13 13:19:19 +00:00
David Robertson
9473ebb9e7
Revert "Fix event size checks (#13710)"
This reverts commit fab495a9e1.

As noted in
https://github.com/matrix-org/synapse/pull/13710#issuecomment-1298396007:

> We want to see this change land for the protocol's sake (and plan to
  un-revert it) but want to give this a little more time before releasing
  this.
2022-11-01 11:47:09 +00:00
DeepBlueV7.X
fab495a9e1
Fix event size checks (#13710) 2022-10-21 09:49:47 +01:00
Shay
847e2393f3
Prepatory work for adding power level event to batched events (#14214) 2022-10-18 09:58:47 -07:00
reivilibre
c2fe48a6ff
Rename the EventFormatVersions enum values so that they line up with room version numbers. (#13706) 2022-09-07 11:08:20 +01:00
Will Hunt
502f075e96
Implement MSC3848: Introduce errcodes for specific event sending failures (#13343)
Implements MSC3848
2022-07-27 13:44:40 +01:00
Shay
ad5761b65c
Add support for room version 10 (#13220) 2022-07-13 11:36:02 -07:00
Richard van der Hoff
d4b1c0d800
Fix inconsistencies in event validation (#13088) 2022-06-17 16:30:59 +01:00
Richard van der Hoff
e16ea87d0f
Fix inconsistencies in event validation for m.room.create events (#13087)
* Extend the auth rule checks for `m.room.create` events

... and move them up to the top of the function. Since the no auth_events are
allowed for m.room.create events, we may as well get the m.room.create event
checks out of the way first.

* Add a test for create events with prev_events
2022-06-17 13:56:46 +00:00
Richard van der Hoff
8ecf6be1e1
Move some event auth checks out to a different method (#13065)
* Add auth events to events used in tests

* Move some event auth checks out to a different method

Some of the event auth checks apply to an event's auth_events, rather than the
state at the event - which means they can play no part in state
resolution. Move them out to a separate method.

* Rename check_auth_rules_for_event

Now it only checks the state-dependent auth rules, it needs a better name.
2022-06-15 19:48:22 +01:00
Richard van der Hoff
0d9d36b15c Remove room_version param from check_auth_rules_for_event
Instead, use the `room_version` property of the event we're checking.

The `room_version` was originally added as a parameter somewhere around #4482,
but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
2022-06-12 23:13:10 +01:00
Richard van der Hoff
68be42f6b6 Remove room_version param from validate_event_for_room_version
Instead, use the `room_version` property of the event we're validating.

The `room_version` was originally added as a parameter somewhere around #4482,
but really it's been redundant since #6875 added a `room_version` field to `EventBase`.
2022-06-12 23:13:09 +01:00
Travis Ralston
942c30b16b
Add a new room version for MSC3787's knock+restricted join rule (#12623) 2022-05-17 10:41:39 +00:00
Travis Ralston
e3fe6347be
Remove excess condition on knock->leave check (#11900) 2022-02-22 18:35:01 +00:00
Richard van der Hoff
4ef39f3353
fix import cycle (#11965) 2022-02-11 13:07:55 +00:00
Patrick Cloke
a121507cfe
Adds misc missing type hints (#11953) 2022-02-11 07:20:16 -05:00
Richard van der Hoff
a5d2ea3d08
Check *all* auth events for room id and rejection (#11009)
This fixes a bug where we would accept an event whose `auth_events` include
rejected events, if the rejected event was shadowed by another `auth_event`
with same `(type, state_key)`.

The approach is to pass a list of auth events into
`check_auth_rules_for_event` instead of a dict, which of course means updating
the call sites.

This is an extension of #10956.
2021-10-18 18:28:30 +01:00
Richard van der Hoff
3a5b0cbe7a
Ensure that we reject events which use rejected events for auth (#10956)
When we consider whether to accept events, we should not accept those which
depend on rejected events for their auth events.

This (together with earlier changes such as
https://github.com/matrix-org/synapse/pull/10771 and
https://github.com/matrix-org/synapse/pull/10896) forms a partial fix to
https://github.com/matrix-org/synapse/issues/9595. There still remain code
paths where we do not check the `auth_events` at all.
2021-10-05 13:23:29 +01:00
Patrick Cloke
d1bf5f7c9d
Strip "join_authorised_via_users_server" from join events which do not need it. (#10933)
This fixes a "Event not signed by authorising server" error when
transition room member from join -> join, e.g. when updating a
display name or avatar URL for restricted rooms.
2021-09-30 11:13:59 -04:00
Richard van der Hoff
428174f902
Split event_auth.check into two parts (#10940)
Broadly, the existing `event_auth.check` function has two parts:
 * a validation section: checks that the event isn't too big, that it has the rught signatures, etc. 
   This bit is independent of the rest of the state in the room, and so need only be done once 
   for each event.
 * an auth section: ensures that the event is allowed, given the rest of the state in the room.
   This gets done multiple times, against various sets of room state, because it forms part of
   the state res algorithm.

Currently, this is implemented with `do_sig_check` and `do_size_check` parameters, but I think
that makes everything hard to follow. Instead, we split the function in two and call each part
separately where it is needed.
2021-09-29 18:59:15 +01:00
Patrick Cloke
c3ccad7785
Only do restricted join rules signature checks for room versions 8/9. (#10927)
Otherwise the presence of a (bogus, unused) field could cause
auth checks to fail.
2021-09-28 08:44:19 -04:00
Eric Eastwood
51e2db3598
Rename MSC2716 things from chunk to batch to match /batch_send endpoint (#10838)
See https://github.com/matrix-org/matrix-doc/pull/2716#discussion_r684574497

Dropping support for older MSC2716 room versions so we don't have to worry about
supporting both chunk and batch events.
2021-09-21 15:06:28 -05:00
Patrick Cloke
0288e6033b
Add a constant for m.federate. (#10775) 2021-09-08 10:00:43 -04:00
Andrew Morgan
7bb3673f37
Ease searching for M_TOO_LARGE-related error codes (#10750) 2021-09-06 14:35:56 +01:00
Patrick Cloke
0c246dd4a0
Support MSC3289: Room version 8 (#10449)
This adds support for MSC3289: room version 8. This is room version 7 + MSC3083.
2021-08-09 10:46:39 +02:00
reivilibre
c167e09fe5
Fix explicit assignment of PL 0 from being misinterpreted in rare circumstances (#10499) 2021-07-30 12:34:21 +01:00
Eric Eastwood
d0b294ad97
Make historical events discoverable from backfill for servers without any scrollback history (MSC2716) (#10245)
* Make historical messages available to federated servers

Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716

Follow-up to https://github.com/matrix-org/synapse/pull/9247

* Debug message not available on federation

* Add base starting insertion point when no chunk ID is provided

* Fix messages from multiple senders in historical chunk

Follow-up to https://github.com/matrix-org/synapse/pull/9247

Part of MSC2716: https://github.com/matrix-org/matrix-doc/pull/2716

---

Previously, Synapse would throw a 403,
`Cannot force another user to join.`,
because we were trying to use `?user_id` from a single virtual user
which did not match with messages from other users in the chunk.

* Remove debug lines

* Messing with selecting insertion event extremeties

* Move db schema change to new version

* Add more better comments

* Make a fake requester with just what we need

See https://github.com/matrix-org/synapse/pull/10276#discussion_r660999080

* Store insertion events in table

* Make base insertion event float off on its own

See https://github.com/matrix-org/synapse/pull/10250#issuecomment-875711889

Conflicts:
	synapse/rest/client/v1/room.py

* Validate that the app service can actually control the given user

See https://github.com/matrix-org/synapse/pull/10276#issuecomment-876316455

Conflicts:
	synapse/rest/client/v1/room.py

* Add some better comments on what we're trying to check for

* Continue debugging

* Share validation logic

* Add inserted historical messages to /backfill response

* Remove debug sql queries

* Some marker event implemntation trials

* Clean up PR

* Rename insertion_event_id to just event_id

* Add some better sql comments

* More accurate description

* Add changelog

* Make it clear what MSC the change is part of

* Add more detail on which insertion event came through

* Address review and improve sql queries

* Only use event_id as unique constraint

* Fix test case where insertion event is already in the normal DAG

* Remove debug changes

* Switch to chunk events so we can auth via power_levels

Previously, we were using `content.chunk_id` to connect one
chunk to another. But these events can be from any `sender`
and we can't tell who should be able to send historical events.
We know we only want the application service to do it but these
events have the sender of a real historical message, not the
application service user ID as the sender. Other federated homeservers
also have no indicator which senders are an application service on
the originating homeserver.

So we want to auth all of the MSC2716 events via power_levels
and have them be sent by the application service with proper
PL levels in the room.

* Switch to chunk events for federation

* Add unstable room version to support new historical PL

* Fix federated events being rejected for no state_groups

Add fix from https://github.com/matrix-org/synapse/pull/10439
until it merges.

* Only connect base insertion event to prev_event_ids

Per discussion with @erikjohnston,
https://matrix.to/#/!UytJQHLQYfvYWsGrGY:jki.re/$12bTUiObDFdHLAYtT7E-BvYRp3k_xv8w0dUQHibasJk?via=jki.re&via=matrix.org

* Make it possible to get the room_version with txn

* Allow but ignore historical events in unsupported room version

See https://github.com/matrix-org/synapse/pull/10245#discussion_r675592489

We can't reject historical events on unsupported room versions because homeservers without knowledge of MSC2716 or the new room version don't reject historical events either.

Since we can't rely on the auth check here to stop historical events on unsupported room versions, I've added some additional checks in the processing/persisting code (`synapse/storage/databases/main/events.py` ->  `_handle_insertion_event` and `_handle_chunk_event`). I've had to do some refactoring so there is method to fetch the room version by `txn`.

* Move to unique index syntax

See https://github.com/matrix-org/synapse/pull/10245#discussion_r675638509

* High-level document how the insertion->chunk lookup works

* Remove create_event fallback for room_versions

See https://github.com/matrix-org/synapse/pull/10245/files#r677641879

* Use updated method name
2021-07-28 10:46:37 -05:00
Patrick Cloke
228decfce1
Update the MSC3083 support to verify if joins are from an authorized server. (#10254) 2021-07-26 12:17:00 -04:00
Jonathan de Jong
bf72d10dbf
Use inline type hints in various other places (in synapse/) (#10380) 2021-07-15 11:02:43 +01:00
Patrick Cloke
eb3beb8f12
Add type hints and comments to event auth code. (#10393) 2021-07-14 14:13:40 +01:00
Patrick Cloke
aaf7d1acb8
Correct type hints for synapse.event_auth. (#10253) 2021-06-30 07:08:42 -04:00
Sorunome
d936371b69
Implement knock feature (#6739)
This PR aims to implement the knock feature as proposed in https://github.com/matrix-org/matrix-doc/pull/2403

Signed-off-by: Sorunome mail@sorunome.de
Signed-off-by: Andrew Morgan andrewm@element.io
2021-06-09 19:39:51 +01:00
Richard van der Hoff
3ff2251754
Improved validation for received requests (#9817)
* Simplify `start_listening` callpath

* Correctly check the size of uploaded files
2021-04-23 19:20:44 +01:00
Patrick Cloke
e83627926f
Add type hints to auth and auth_blocking. (#9876) 2021-04-23 12:02:16 -04:00
Jonathan de Jong
495b214f4f
Fix (final) Bugbear violations (#9838) 2021-04-20 11:50:49 +01:00
Jonathan de Jong
4b965c862d
Remove redundant "coding: utf-8" lines (#9786)
Part of #9744

Removes all redundant `# -*- coding: utf-8 -*-` lines from files, as python 3 automatically reads source code as utf-8 now.

`Signed-off-by: Jonathan de Jong <jonathan@automatia.nl>`
2021-04-14 15:34:27 +01:00
Patrick Cloke
35c5ef2d24
Add an experimental room version to support restricted join rules. (#9717)
Per MSC3083.
2021-03-31 16:39:08 -04:00
Eric Eastwood
0a00b7ff14
Update black, and run auto formatting over the codebase (#9381)
- Update black version to the latest
 - Run black auto formatting over the codebase
    - Run autoformatting according to [`docs/code_style.md
`](80d6dc9783/docs/code_style.md)
 - Update `code_style.md` docs around installing black to use the correct version
2021-02-16 22:32:34 +00:00
Patrick Cloke
da11cc22be
Ensure that event.redacts is the proper type before handling it (#8457)
This fixes a bug when backfilling invalid events.
2020-10-05 10:24:17 -04:00
Patrick Cloke
f40645e60b
Convert events worker database to async/await. (#8071) 2020-08-18 16:20:49 -04:00
Erik Johnston
e29c44340b
Fix recursion error when fetching auth chain over federation (#7817)
When fetching the state of a room over federation we receive the event
IDs of the state and auth chain. We then fetch those events that we
don't already have.

However, we used a function that recursively fetched any missing auth
events for the fetched events, which can lead to a lot of recursion if
the server is missing most of the auth chain. This work is entirely
pointless because would have queued up the missing events in the auth
chain to be fetched already.

Let's just diable the recursion, since it only gets called from one
place anyway.
2020-07-10 18:15:35 +01:00
Patrick Cloke
5355421295
Add type hints to event_auth code. (#7505) 2020-05-15 11:19:43 -04:00