Commit Graph

20483 Commits

Author SHA1 Message Date
Shay
f608e6c8cf
Update broken link in workers.md (#12369) 2022-04-04 17:40:31 -07:00
Eric Eastwood
9633eb2162
Allow non-member state sent in room batch to resolve for historic events (MSC2716) (#12329)
Part of https://github.com/matrix-org/synapse/issues/12110

Complement test: https://github.com/matrix-org/complement/pull/354

Previously, they didn't resolve because async `filter_events_for_client`
removes all outlier state except for out-of-band membership.

And fundamentally, we have the state at these events so they shouldn't be marked as outliers.
2022-04-04 14:54:50 -05:00
Erik Johnston
b446c99ac9
Prefill the device_list_stream_cache (#12367)
* Prefill the device_list_stream_cache

* Newsfile

* Newsfile
2022-04-04 20:12:25 +01:00
Erik Johnston
5c9e39e619
Track device list updates per room. (#12321)
This is a first step in dealing with #7721.

The idea is basically that rather than calculating the full set of users a device list update needs to be sent to up front, we instead simply record the rooms the user was in at the time of the change. This will allow a few things:

1. we can defer calculating the set of remote servers that need to be poked about the change; and
2. during `/sync` and `/keys/changes` we can avoid also avoid calculating users who share rooms with other users, and instead just look at the rooms that have changed.

However, care needs to be taken to correctly handle server downgrades. As such this PR writes to both `device_lists_changes_in_room` and the `device_lists_outbound_pokes` table synchronously. In a future release we can then bump the database schema compat version to `69` and then we can assume that the new `device_lists_changes_in_room` exists and is handled.

There is a temporary option to disable writing to `device_lists_outbound_pokes` synchronously, allowing us to test the new code path does work (and by implication upgrading to a future release and downgrading to this one will work correctly).

Note: Ideally we'd do the calculation of room to servers on a worker (e.g. the background worker), but currently only master can write to the `device_list_outbound_pokes` table.
2022-04-04 15:25:20 +01:00
David Robertson
80839a44f1
Remove more dead/broken dev scripts (#12355) 2022-04-03 21:21:08 +01:00
Richard van der Hoff
f0b03186d9
Add type hints for tests/unittest.py. (#12347)
In particular, add type hints for get_success and friends, which are then helpful in a bunch of places.
2022-04-01 16:04:16 +00:00
Richard van der Hoff
33ebee47e4
Remove redundant get_success calls in test code (#12346)
There are a bunch of places we call get_success on an immediate value, which is unnecessary. Let's rip them out, and remove the redundant functionality in get_success and friends.
2022-04-01 16:10:31 +01:00
reivilibre
c4cf916ed7
Default to private room visibility rather than public when a client does not specify one, according to spec. (#12350) 2022-04-01 15:55:09 +01:00
David Robertson
336bff1104
Burn check_signature dev script. (#12351) 2022-04-01 13:41:42 +00:00
Nick Mills-Barrett
993d90f82b
Use a sequence to generate AS transaction IDs, drop last_txn AS state (#12209)
Switching to a sequence means there's no need to track `last_txn` on the
AS state table to generate new TXN IDs. This also means that there is
no longer contention between the AS scheduler and AS handler on updates
to the `application_services_state` table, which will prevent serialization
errors during the complete AS txn transaction.
2022-04-01 13:33:25 +00:00
David Robertson
21351820e0
Remove list_url_patterns dev script (#12349) 2022-04-01 13:05:21 +00:00
Richard van der Hoff
b7762b0c9f
Move single-use methods out of TestCase (#12348)
These methods are only used by a single testcase, so they shouldn't be
cluttering up the base `TestCase` class.
2022-04-01 12:48:08 +00:00
reivilibre
f871222880
Move update_client_ip background job from the main process to the background worker. (#12251) 2022-04-01 13:08:55 +01:00
Richard van der Hoff
319a805cd3
Raise an exception when getting state at an outlier (#12191)
It seems like calling `_get_state_group_for_events` for an event where the
state is unknown is an error. Accordingly, let's raise an exception rather than
silently returning an empty result.
2022-04-01 13:01:49 +01:00
Richard van der Hoff
9b43df1f7b
Optimise _get_state_after_missing_prev_event: use /state (#12040)
If we're missing most of the events in the room state, then we may as well call the /state endpoint, instead of individually requesting each and every event.
2022-04-01 12:53:42 +01:00
Brendan Abolivier
e4409301ba
Add a module callback to react to account data changes (#12327)
Co-authored-by: Sean Quah <8349537+squahtx@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2022-04-01 11:22:48 +02:00
Amanda Graven
4e900ece42
Add set_user_admin function to the module API (#12341) 2022-04-01 09:31:30 +01:00
Patrick Cloke
bebf994ee8
Move MSC2654 support behind an experimental configuration flag. (#12295)
To match the current thinking on disabling experimental features by default.
2022-03-31 15:05:13 -04:00
Richard van der Hoff
6927d87254
Handle outliers in /federation/v1/event (#12332)
The intention here is to avoid doing state lookups for outliers in
`/_matrix/federation/v1/event`. Unfortunately that's expanded into something of
a rewrite of `filter_events_for_server`, which ended up trying to do that
operation in a couple of places.
2022-03-31 17:39:34 +00:00
Patrick Cloke
11df4ec6c2
Add more type hints to the main state store. (#12267) 2022-03-31 13:38:09 -04:00
Brendan Abolivier
5e88143dff
Add a callback to react to 3PID associations (#12302) 2022-03-31 18:27:21 +02:00
Richard van der Hoff
34a8370d7b
README-testing.md: fix minor error 2022-03-31 13:08:03 +01:00
Patrick Cloke
adbf975623
Remove an unnecessary class from the relations code. (#12338)
The PaginationChunk class attempted to bundle some properties
together, but really just caused callers to jump through hoops and
hid implementation details.
2022-03-31 07:13:49 -04:00
David Robertson
15cdcf8f30
Remove dockerfile-pgtests (#12336) 2022-03-31 11:30:07 +01:00
David Robertson
5a32ec59b2
Ignore .envrc for direnv users (#12335) 2022-03-31 11:03:07 +01:00
David Robertson
9a3f1f5383
Nuke the tox packaging job (#12334) 2022-03-31 11:01:26 +01:00
Brendan Abolivier
f96b85eca8
Ensure the type of URL attributes is always str when matching against preview blacklist (#12333) 2022-03-31 11:49:49 +02:00
Patrick Cloke
c31c1091d4
Remove the unused and unstable /aggregations endpoint. (#12293)
This endpoint was removed from MSC2675 before it was approved.
It is currently unspecified (even in any MSCs) and therefore subject to
removal. It is not implemented by any known clients.

This also changes the bundled aggregation format for `m.annotation`,
which previously included pagination tokens for the `/aggregations`
endpoint, which are no longer useful.
2022-03-30 11:45:32 -04:00
Andrew Morgan
d8d0271977
Send device list updates to application services (MSC3202) - part 1 (#11881)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-30 14:39:27 +01:00
reivilibre
2fc15ac718
Flesh out documentation for running SyTest against Synapse, including use of Postgres and worker mode. (#12271)
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-03-30 12:55:05 +00:00
David Robertson
a7fb66e800
Update dead links in check-newsfragment.sh (#12331) 2022-03-30 11:44:28 +00:00
Andrew Morgan
19a1d6a42a
Remove references to "msc2403" (#12165) 2022-03-30 11:15:47 +00:00
Andrew Morgan
c8cbd66d3b
Start application service stream token tracking from 1 (#12193)
Co-authored-by: Erik Johnston <erik@matrix.org>
2022-03-30 11:10:12 +00:00
Richard van der Hoff
9b67715bc3
Disable proactive sends for remote joins (#12330)
Do not attempt to send remote joins out over federation. Normally, it will do
nothing; occasionally, it will do the wrong thing.
2022-03-30 12:04:35 +01:00
Brendan Abolivier
437a8ed9ef
Add a configuration to exclude rooms from sync response (#12310) 2022-03-30 09:43:04 +00:00
David Robertson
e0bb268134
Fix typechecker problems exposed by signedjson 1.1.2 (#12326) 2022-03-29 21:37:50 +00:00
Richard van der Hoff
1f32b90b0f
Room batch: fix up handling of unknown prev_event_ids (#12316) 2022-03-29 14:56:25 +01:00
Andrew Morgan
4d693f9b79
Add note in changelog of non-C type locale causing Synapse to refuse to start (#12323) 2022-03-29 14:32:50 +01:00
Andrew Morgan
013f3f5e44
Move warning of open registration to v1.56 upgrade notes (#12322) 2022-03-29 14:12:41 +01:00
Sean Quah
8a519f8abc
Update LoggingTransaction.call_after and call_on_exception docstrings (#12315)
Document the behaviour of `LoggingTransaction.call_after` and
`LoggingTransaction.call_on_exception` when transactions are retried.

Signed-off-by: Sean Quah <seanq@element.io>
2022-03-29 12:31:05 +01:00
David Robertson
a2b00a4486
Bump black and click versions (#12320) 2022-03-29 10:41:19 +00:00
Brendan Abolivier
8a5d691140
Missing period 2022-03-29 11:28:39 +01:00
Brendan Abolivier
512007f829
Fix formatting 2022-03-29 11:25:49 +01:00
Brendan Abolivier
e9220adffc
Link to pre-1.0 changelog 2022-03-29 11:25:06 +01:00
Brendan Abolivier
28a64807b2
Linkify MSCs 2022-03-29 11:22:54 +01:00
Brendan Abolivier
d653f6fbec
Split changelogs 2022-03-29 11:21:11 +01:00
Brendan Abolivier
c20d0ca6c2
Fixup changelog 2022-03-29 10:50:35 +01:00
Brendan Abolivier
b690fe749b 1.56.0rc1 2022-03-29 10:41:04 +01:00
Eric Eastwood
6f2943714b
Remove unused auth_event_ids argument plumbing (#12304)
Follow-up to https://github.com/matrix-org/synapse/pull/12083

Since we are now using the new `state_event_ids` parameter to do all of the heavy lifting.
We can remove any spots where we plumbed `auth_event_ids` just for MSC2716 things in
https://github.com/matrix-org/synapse/pull/9247/files.

Removing `auth_event_ids` from following functions:

 - `create_and_send_nonmember_event`
 - `_local_membership_update`
 - `update_membership`
 - `update_membership_locked`
2022-03-29 09:18:52 +01:00
Richard van der Hoff
287a9c1e20
Exclude outliers in on_backfill_request (#12314)
When we are processing a `/backfill` request from a remote server, exclude any
outliers from consideration early on. We can't return outliers anyway (since we
don't know the state at the outlier), and filtering them out earlier means that
we won't attempt to calulate the state for them.
2022-03-28 19:20:14 +01:00