Commit Graph

21169 Commits

Author SHA1 Message Date
Jörg Behrmann
87a917e8c8
Add notes when config options were changed to config documentation ()
Signed-off-by: Jörg Behrmann <behrmann@physik.fu-berlin.de>
2022-07-19 12:36:29 +00:00
David Robertson
b977867358
Rate limit joins per-room () 2022-07-19 11:45:17 +00:00
Nick Mills-Barrett
2ee0b6ef4b
Safe async event cache ()
Fix race conditions in the async cache invalidation logic, by separating
the async & local invalidation calls and ensuring any async call i
executed first.

Signed off by Nick @ Beeper (@Fizzadar).
2022-07-19 11:25:29 +00:00
Shay
7864f33e28
Increase batch size of bulk_get_push_rules and _get_joined_profiles_from_event_ids. () 2022-07-18 13:15:23 -07:00
Shay
15edf23626
Improve performance of query _get_subset_users_in_room_with_profiles () 2022-07-18 12:35:45 -07:00
Sean Quah
5526f9fc4f
Fix overcounting of pushers when they are replaced ()
Signed-off-by: Sean Quah <seanq@matrix.org>
2022-07-18 17:39:39 +01:00
Brendan Abolivier
8c60c572f0
Up the dependency on canonicaljson to ^1.5.0 ()
Co-authored-by: David Robertson <davidr@element.io>
2022-07-18 17:30:59 +02:00
Andrew Morgan
bb25dd81e3
Prevent from appearing in blame results () 2022-07-18 14:02:32 +00:00
Erik Johnston
f721f1baba
Revert "Make all process_replication_rows methods async ()" ()
This reverts commit 5d4028f217.
2022-07-18 14:28:14 +01:00
Erik Johnston
cf5fa5063d
Don't pull out full state when sending dummy events () 2022-07-18 14:19:11 +01:00
Nick Mills-Barrett
6785b0f39d
Use READ COMMITTED isolation level when purging rooms ()
To close: .

Signed off by Nick @ Beeper.
2022-07-18 14:17:24 +01:00
Andrew Morgan
c5f487b7cb
Update expected DB query count when creating a room () 2022-07-18 13:02:25 +01:00
Erik Johnston
c6a05063ff
Don't pull out the full state when creating an event () 2022-07-18 10:05:30 +01:00
Dirk Klimpel
efee345b45
Remove unnecessary json.dumps from tests () 2022-07-17 22:28:45 +01:00
Nick Mills-Barrett
5d4028f217
Make all process_replication_rows methods async ()
More prep work for asyncronous caching, also makes all process_replication_rows methods consistent (presence handler already is so).

Signed off by Nick @ Beeper (@Fizzadar)
2022-07-17 22:19:43 +01:00
Dirk Klimpel
96cf81e312
Use HTTPStatus constants in place of literals in tests. () 2022-07-15 19:31:27 +00:00
Eric Eastwood
7b67e93d49
Provide more info why we don't have any thumbnails to serve ()
Fix https://github.com/matrix-org/synapse/issues/13016

## New error code and status

### Before

Previously, we returned a `404` for `/thumbnail` which isn't even in the spec.

```json
{
  "errcode": "M_NOT_FOUND",
  "error": "Not found [b'hs1', b'tefQeZhmVxoiBfuFQUKRzJxc']"
}
```

### After

What does the spec say?

> 400: The request does not make sense to the server, or the server cannot thumbnail the content. For example, the client requested non-integer dimensions or asked for negatively-sized images.
>
> *-- https://spec.matrix.org/v1.1/client-server-api/#get_matrixmediav3thumbnailservernamemediaid*

Now with this PR, we respond with a `400` when we don't have thumbnails to serve and we explain why we might not have any thumbnails.

```json
{
    "errcode": "M_UNKNOWN",
    "error": "Cannot find any thumbnails for the requested media ([b'example.com', b'12345']). This might mean the media is not a supported_media_format=(image/jpeg, image/jpg, image/webp, image/gif, image/png) or that thumbnailing failed for some other reason. (Dynamic thumbnails are disabled on this server.)",
}
```

> Cannot find any thumbnails for the requested media ([b'example.com', b'12345']). This might mean the media is not a supported_media_format=(image/jpeg, image/jpg, image/webp, image/gif, image/png) or that thumbnailing failed for some other reason. (Dynamic thumbnails are disabled on this server.)


---

We still respond with a 404 in many other places. But we can iterate on those later and maybe keep some in some specific places after spec updates/clarification: https://github.com/matrix-org/matrix-spec/issues/1122

We can also iterate on the bugs where Synapse doesn't thumbnail when it should in other issues/PRs.
2022-07-15 11:42:21 -05:00
David Robertson
e9ce4d089b
Use and recommend poetry 1.1.14, up from 1.1.12 () 2022-07-15 16:18:47 +01:00
Erik Johnston
0731e0829c
Don't pull out the full state when storing state () 2022-07-15 12:59:45 +00:00
Patrick Cloke
3343035a06
Use a real room in the notification rotation tests. ()
Instead of manually inserting fake data. This fixes some issues with
having to manually calculate stream orderings and other oddities.
2022-07-15 08:22:43 -04:00
David Robertson
7281591f4c
Use state before join to determine if we _should_perform_remote_join ()
Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2022-07-15 12:20:47 +00:00
Sean Quah
d765ada84f
Update locked frozendict version to 2.3.2 ()
`frozendict` 2.3.2 includes a fix for a memory leak in
`frozendict.__hash__`. This likely has no impact outside of the
deprecated `/initialSync` endpoint, which uses `StreamToken`s,
containing `RoomStreamToken`s, containing `frozendict`s, as cache keys.

Signed-off-by: Sean Quah <seanq@matrix.org>
2022-07-15 13:18:51 +01:00
Richard van der Hoff
b116d3ce00
Bg update to populate new events table columns ()
These columns were added back in Synapse 1.52, and have been populated for new
events since then. It's now (beyond) time to back-populate them for existing
events.
2022-07-15 12:47:26 +01:00
Erik Johnston
7be954f59b
Fix a bug which could lead to incorrect state ()
There are two fixes here:
1. A long-standing bug where we incorrectly calculated `delta_ids`; and
2. A bug introduced in  where we got current state incorrect.
2022-07-15 11:06:41 +00:00
Richard van der Hoff
512486bbeb
Docker: copy postgres from base image ()
When building the docker images for complement testing, copy a preinstalled
complement over from a base image, rather than apt installing it. This avoids
network traffic and is much faster.
2022-07-15 11:13:40 +01:00
Nick Mills-Barrett
cc21a431f3
Async get event cache prep ()
Some experimental prep work to enable external event caching based on  & . Doesn't actually move the cache at all, just lays the groundwork for async implemented caches.

Signed off by Nick @ Beeper (@Fizzadar)
2022-07-15 09:30:46 +00:00
Nick Mills-Barrett
21eeacc995
Federation Sender & Appservice Pusher Stream Optimisations ()
* Replace `get_new_events_for_appservice` with `get_all_new_events_stream`

The functions were near identical and this brings the AS worker closer
to the way federation senders work which can allow for multiple workers
to handle AS traffic.

* Pull received TS alongside events when processing the stream

This avoids an extra query -per event- when both federation sender
and appservice pusher process events.
2022-07-15 09:36:56 +01:00
Richard van der Hoff
fe15a865a5
Rip out auth-event reconciliation code ()
There is a corner in `_check_event_auth` (long known as "the weird corner") where, if we get an event with auth_events which don't match those we were expecting, we attempt to resolve the diffence between our state and the remote's with a state resolution.

This isn't specced, and there's general agreement we shouldn't be doing it.

However, it turns out that the faster-joins code was relying on it, so we need to introduce something similar (but rather simpler) for that.
2022-07-14 21:52:26 +00:00
Richard van der Hoff
df55b377be
CHANGES.md: fix link to upgrade notes 2022-07-14 15:07:52 +01:00
Erik Johnston
0ca4172b5d
Don't pull out state in compute_event_context for unconflicted state () 2022-07-14 13:57:02 +00:00
David Robertson
599c403d99
Allow rate limiters to passively record actions they cannot limit ()
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-07-13 19:09:42 +00:00
David Robertson
0eb7e69768
Notifier: accept callbacks to fire on room joins () 2022-07-13 19:48:24 +01:00
Jacek Kuśnierz
cc1071598a
Call the v2 identity service /3pid/unbind endpoint, rather than v1. ()
* Drop support for v1 unbind

Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>

* Add changelog

Signed-off-by: Jacek Kusnierz <jacek.kusnierz@tum.de>

* Update changelog.d/13240.misc
2022-07-13 19:43:17 +01:00
Shay
ad5761b65c
Add support for room version 10 () 2022-07-13 11:36:02 -07:00
jejo86
2341032cf2
Document advising against publicly exposing the Admin API and provide a usage example ()
* Admin API request explanation improved

Pointed out, that the Admin API is not accessible by default from any remote computer, but only from the PC `matrix-synapse` is running on.
Added a full, working example, making sure to include the cURL flag `-X`, which needs to be prepended to `GET`, `POST`, `PUT` etc. and listing the full query string including protocol, IP address and port.

* Admin API request explanation improved

* Apply suggestions from code review

Update changelog. Reword prose.

Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-07-13 19:33:33 +01:00
Nick Mills-Barrett
982fe29655
Optimise room creation event lookups part 2 () 2022-07-13 19:32:46 +01:00
Patrick Cloke
1d5c80b161
Reduce duplicate code in receipts servlets. () 2022-07-13 13:23:16 -04:00
Brad Murray
3371e1abcb
Add prometheus counters for content types other than events () 2022-07-13 15:18:20 +01:00
Patrick Cloke
4db7862e0f
Drop unused tables from groups/communities. ()
These tables have been unused since Synapse v1.61.0, although schema version 72
was added in Synapse v1.62.0.
2022-07-13 09:55:14 -04:00
Patrick Cloke
90e9b4fa1e
Do not fail build if complement with workers fails. () 2022-07-13 08:30:42 -04:00
Thomas Weston
0312ff44c6
Fix "add user" admin api error when request contains a "msisdn" threepid ()
Co-authored-by: Thomas Weston <thomas.weston@clearspancloud.com>
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-07-13 11:33:21 +01:00
Patrick Cloke
1381563988
Inline URL preview documentation. ()
Inline URL preview documentation near the implementation.
2022-07-12 15:01:58 -04:00
Richard van der Hoff
a366b75b72
Drop unused table event_reference_hashes ()
This is unused since Synapse 1.60.0 (). It's time for it to go.
2022-07-12 18:52:06 +00:00
Jacek Kuśnierz
7218a0ca18
Drop support for calling /_matrix/client/v3/account/3pid/bind without an id_access_token ()
Fixes 

Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de
2022-07-12 18:48:29 +00:00
David Robertson
52a0c8f2f7
Rename test case method to add_hashes_and_signatures_from_other_server () 2022-07-12 18:46:32 +00:00
Richard van der Hoff
fa71bb18b5
Drop support for delegating email validation ()
* Drop support for delegating email validation

Delegating email validation to an IS is insecure (since it allows the owner of
the IS to do a password reset on your HS), and has long been deprecated. It
will now cause a config error at startup.

* Update unit test which checks for email verification

Give it an `email` config instead of a threepid delegate

* Remove unused method `requestEmailToken`

* Simplify config handling for email verification

Rather than an enum and a boolean, all we need here is a single bool, which
says whether we are or are not doing email verification.

* update docs

* changelog

* upgrade.md: fix typo

* update version number

this will be in 1.64, not 1.63

* update version number

this one too
2022-07-12 19:18:53 +01:00
Sean Quah
3f178332d6
Log the stack when waiting for an entire room to be un-partial stated ()
The stack is already logged when waiting for an event to be un-partial
stated. Log the stack for rooms as well, to aid in debugging.
2022-07-12 18:57:38 +01:00
Shay
6f30eb5b8e
Add info about configuration in the url preview docs ()
Cross-link doc pages for easier navigation.
2022-07-12 13:48:47 -04:00
Quentin Gliech
b19060a29b
Make the AS login method call Auth.get_user_by_req for checking the AS token. ()
This gets rid of another usage of get_appservice_by_req, with all the benefits, including correctly tracking the appservice IP and setting the tracing attributes correctly.

Signed-off-by: Quentin Gliech <quenting@element.io>
2022-07-12 18:06:29 +01:00
andrew do
2d82cdafd2
expose whether a room is a space in the Admin API () 2022-07-12 15:30:53 +01:00