Commit Graph

8145 Commits

Author SHA1 Message Date
Erik Johnston
3e6ee8ff88
Add optimisation to StreamChangeCache (#17130)
When there have been lots of changes compared with the number of
entities, we can do a fast(er) path.

Locally I ran some benchmarking, and the comparison seems to give the
best determination of which method we use.
2024-05-06 12:56:52 +01:00
Erik Johnston
7c9ac01eb5
Fix bug where StreamChangeCache would not respect cache factors (#17152)
Annoyingly mypy didn't pick up this typo.
2024-05-03 18:00:08 +01:00
Erik Johnston
3818597751
Fix lint.sh script (#17148)
Broke in #17073
2024-05-03 17:12:03 +01:00
Andrew Morgan
3aadf43122
Bump pillow from 10.2.0 to 10.3.0 (#17146) 2024-05-03 10:55:59 +01:00
jahway603
5b6a75935e
upgrade.md: Bump minimum Rust version to 1.66.0 (element-hq#17079) (#17140)
upgrade.md: Bump minimum Rust version to 1.66.0 (element-hq#17079)
2024-05-02 14:57:29 +01:00
Benjamin Bouvier
c0ea2bf800
synapse complement image: hardcode enabling msc3266 (#17105)
This is an alternative to
https://github.com/matrix-org/matrix-rust-sdk/issues/3340 where we don't
need to change our CI setup too much in the Rust SDK repository, and
still can test MSC3266.
2024-05-02 11:48:27 +01:00
Shay
37558d5e4c
Add support for MSC3823 - Account Suspension (#17051) 2024-05-01 17:45:17 +01:00
Erik Johnston
0b358f8643
Drop sphinx docs (#17073)
It is broken, and we only seemed to have been building it for the
federation sender.

Closes https://github.com/element-hq/synapse/issues/16804
2024-05-01 16:01:50 +00:00
Ben Banfield-Zanin
7254015665
Correct licensing metadata on the Docker image (#17141)
### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-05-01 16:23:42 +01:00
Richard van der Hoff
07232e27a8
Enable complement tests for MSC4115 support (#17137)
Follow-up to #17137 and
https://github.com/matrix-org/complement/pull/722
2024-04-30 13:57:20 +01:00
devonh
7ab0f630da
Apply user email & picture during OIDC registration if present & selected (#17120)
This change will apply the `email` & `picture` provided by OIDC to the
new user account when registering a new user via OIDC. If the user is
directed to the account details form, this change makes sure they have
been selected before applying them, otherwise they are omitted. In
particular, this change ensures the values are carried through when
Synapse has consent configured, and the redirect to the consent form/s
are followed.

I have tested everything manually. Including: 
- with/without consent configured
- allowing/not allowing the use of email/avatar (via
`sso_auth_account_details.html`)
- with/without automatic account detail population (by un/commenting the
`localpart_template` option in synapse config).

### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [X] Pull request is based on the develop branch
* [X] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-04-29 15:23:05 +00:00
Richard van der Hoff
b548f7803a
Add support for MSC4115 (#17104)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-29 15:22:13 +01:00
Andrew Morgan
758aec6b34
Update tornado 6.2 -> 6.4 (#17131) 2024-04-29 14:33:25 +01:00
Richard van der Hoff
c897ac63e9
Ensure that incoming to-device messages are not dropped (#17127)
... when workers are unreachable, etc.

Fixes https://github.com/element-hq/synapse/issues/17117.

The general principle is just to make sure that we propagate any
exceptions to the JsonResource, so that we return an error code to the
sending server. That means that the sending server no longer considers
the message safely sent, so it will retry later.

In the issue, Erik mentions that an alternative solution would be to
persist the to-device messages into a table so that they can be retried.
This might be an improvement for performance, but even if we did that,
we still need this mechanism, since we might be unable to reach the
database. So, if we want to do that, it can be a later follow-up.

---------

Co-authored-by: Erik Johnston <erik@matrix.org>
2024-04-29 14:11:00 +01:00
Patrick Cloke
38bc7a009d
Declare support for Matrix v1.10. (#17082)
Pretty straightforward. 😄 

Fixes #17021
2024-04-29 14:09:03 +01:00
devonh
0fd6b269d3
Fix various typos in docs (#17114)
### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [X] Pull request is based on the develop branch
* [X] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-04-26 18:10:45 +00:00
Andrew Morgan
89fc579329
Fix filtering of rooms when supplying the destination query parameter to /_synapse/admin/v1/federation/destinations/<destination>/rooms (#17077) 2024-04-26 10:52:24 +01:00
villepeh
9c91873922
Add RuntimeDirectory to matrix-synapse.service (#17084)
This makes it easy to store UNIX sockets with correct permissions. Those
would be located in /run/synapse which is the directory used in many
examples in Synapse configuration manual. Additionally, the directory
and sockets are deleted when Synapse is shut down.
2024-04-26 09:56:20 +01:00
Michael Telatynski
41fbe387d6
Improve error message for cross signing reset with MSC3861 enabled (#17121) 2024-04-26 09:54:30 +01:00
Amanda H. L. de Andrade Katz
90cc9e5b29
Rephrase enable_notifs configuration (#17116) 2024-04-26 09:52:58 +01:00
Andrew Ferrazzutti
516fd891ee
Use recommended endpoint for MSC3266 requests (#17078)
Keep the existing endpoint for backwards compatibility

Signed-off-by: Andrew Ferrazzutti <andrewf@element.io>
2024-04-26 09:46:42 +01:00
Amanda H. L. de Andrade Katz
0ef2315a99
Update event_cache_size and global_factor configurations documentation (#17071)
### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-04-26 09:44:54 +01:00
Melvyn Laïly
59710437e4
Return the search terms as search highlights for SQLite instead of nothing (#17000)
Fixes https://github.com/element-hq/synapse/issues/16999 and
https://github.com/element-hq/element-android/pull/8729 by returning the
search terms as search highlights.
2024-04-26 09:43:52 +01:00
Olivier 'reivilibre
48a90c697b 1.106.0rc1 2024-04-25 15:55:18 +01:00
Till
47773232b0
Redact membership events if the user requested erasure upon deactivating (#17076)
Fixes #15355 by redacting all membership events before leaving rooms.
2024-04-25 14:25:31 +01:00
Quentin Gliech
2e92b718d5
MSC4108 implementation (#17056)
Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
Co-authored-by: Hugh Nimmo-Smith <hughns@users.noreply.github.com>
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-25 12:50:12 +00:00
Andrew Morgan
646cb6ff24
Add type annotation to visited_chains (#17125)
This should fix CI on `develop`. Broke in
0fe9e1f7da,
presumably due to a `mypy` dependency upgrade.
2024-04-25 12:25:26 +00:00
mcalinghee
ae181233aa
Send an email if the address is already bound to an user account (#16819)
Co-authored-by: Mathieu Velten <mathieu.velten@beta.gouv.fr>
Co-authored-by: Olivier D <odelcroi@gmail.com>
2024-04-23 16:45:24 +01:00
Neil Johnson
074ef4d75f
Add an OSX prompt to manually configure icu4c. (#17069)
Documentation fix.
2024-04-19 17:10:44 +01:00
devonh
301c9771c4
Clarify what part of message retention is still experimental (#17099)
### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [X] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [X] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))
2024-04-19 15:26:28 +00:00
Gordan Trevis
1d47532310
Parse json validation (#16923)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-18 13:57:38 +01:00
Quentin Gliech
09f0957b36
Helpers to transform Twisted requests to Rust http Requests/Responses (#17081)
This adds functions to transform a Twisted request to the
`http::Request`, and then to send back an `http::Response` through it.

It also imports the SynapseError exception so that we can throw that
from Rust code directly

Example usage of this would be:

```rust
use crate::http::{http_request_from_twisted, http_response_to_twisted, HeaderMapPyExt};

fn handler(twisted_request: &PyAny) -> PyResult<()> {
    let request = http_request_from_twisted(twisted_request)?;

    let ua: headers::UserAgent = request.headers().typed_get_required()?;

    if whatever {
        return Err((crate::errors::SynapseError::new(
            StatusCode::UNAUTHORIZED,
            "Whatever".to_owned
            "M_UNAUTHORIZED",
            None,
            None,
        )));
    }

    let response = Response::new("hello".as_bytes());
    http_response_to_twisted(twisted_request, response)?;

    Ok(())
}
```
2024-04-18 12:20:30 +02:00
Erik Johnston
803f05f60c
Fix remote receipts for events we don't have (#17096)
Introduced in #17032
2024-04-17 16:08:40 +01:00
Quentin Gliech
c8e0bed426
Support for MSC4108 via delegation (#17086)
This adds support for MSC4108 via delegation, similar to what has been done for MSC3886

---------

Co-authored-by: Hugh Nimmo-Smith <hughns@element.io>
2024-04-17 16:47:35 +02:00
Quentin Gliech
28f5ad07d3
Bump minimum required Rust version to 1.66.0 (#17079) 2024-04-17 15:44:40 +02:00
Gordan Trevis
f0d6f14047
Parse Integer negative value validation (#16920) 2024-04-16 19:12:36 +00:00
Kegan Dougal
259442fa4c
bugfix: make msc3967 idempotent (#16943)
MSC3967 was updated recently to make it more robust to network failures:

> there is an existing cross-signing master key and it exactly matches
the cross-signing master key provided in the request body. If there are
any additional keys provided in the request (self signing key, user
signing key) they MUST also match the existing keys stored on the
server. In other words, the request contains no new keys. If there are
new keys, UIA MUST be performed.


https://github.com/matrix-org/matrix-spec-proposals/blob/hughns/device-signing-upload-uia/proposals/3967-device-signing-upload-uia.md#proposal

This covers the case where the 200 OK is lost in transit so the client
retries the upload, only to then get UIA'd.

Complement tests: https://github.com/matrix-org/complement/pull/713 -
passing example
https://github.com/element-hq/synapse/actions/runs/7976948122/job/21778795094?pr=16943#step:7:8820

### Pull Request Checklist

<!-- Please read
https://element-hq.github.io/synapse/latest/development/contributing_guide.html
before submitting your pull request -->

* [x] Pull request is based on the develop branch
* [x] Pull request includes a [changelog
file](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#changelog).
The entry should:
- Be a short description of your change which makes sense to users.
"Fixed a bug that prevented receiving messages from other servers."
instead of "Moved X method from `EventStore` to `EventWorkerStore`.".
  - Use markdown where necessary, mostly for `code blocks`.
  - End with either a period (.) or an exclamation mark (!).
  - Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by
@github_username." or "Contributed by [Your Name]." to the end of the
entry.
* [x] [Code
style](https://element-hq.github.io/synapse/latest/code_style.html) is
correct
(run the
[linters](https://element-hq.github.io/synapse/latest/development/contributing_guide.html#run-the-linters))

---------

Co-authored-by: reivilibre <oliverw@matrix.org>
2024-04-15 10:57:56 +00:00
Nick Mills-Barrett
fe4719a268
Use receipts event_stream_ordering instead of joins (#17032)
Resurrecting https://github.com/matrix-org/synapse/pull/13918.

This should reduce IOPs incurred by joining to the events table to
lookup stream ordering, which happens in many receipt handling code
paths. Like the previous PR I believe sufficient time has passed between
the original migration in DB schema 72 and now to merge this as-is. It's
highly unlikely that both the migration is still ongoing AND (active)
users still have any receipts prior to that date.

In the unlikely event there is a receipt without a populated
`event_stream_ordering` synapse will behave just as it does now when
receipts exist for events that don't (yet): for push action calculation
the receipts are just ignored.

I've removed the validation on event IDs as this is already covered
here:

59ceabcb97/synapse/handlers/receipts.py (L189-L192)
2024-04-12 09:28:44 +01:00
Erik Johnston
3a30846bd0
Fix mypy on latest Twisted release (#17036)
`ITransport.abortConnection` isn't a thing, but
`HTTPChannel.forceAbortClient` calls it, so lets just use that

Fixes https://github.com/element-hq/synapse/issues/16728
2024-04-11 16:03:45 +01:00
Andrew Morgan
db4e321219 1.105.0rc1 2024-04-11 12:16:31 +01:00
Patrick Cloke
657b8cc75c
Stabilize support for MSC4010: push rules & account data. (#17022)
See
[MSC4010](https://github.com/matrix-org/matrix-spec-proposals/pull/4010),
but this is pretty much just removing an experimental flag.

Part of #17021
2024-04-09 17:11:50 +01:00
Patrick Cloke
a2a543fd12
Stabliize support for MSC3981: recurse /relations (#17023)
See
[MSC3981](https://github.com/matrix-org/matrix-spec-proposals/pull/3981),
this pretty much just removes flags though.

Part of #17021
2024-04-09 17:11:08 +01:00
Erik Johnston
89f1092284
Also check if first event matches the last in prev batch (#17066)
Refinement of #17064 

cc @richvdh
2024-04-09 14:01:12 +00:00
Sumiran Pokharel
4ffed6330f
#17039 Issue: Update base_rules.rs (#17043)
Co-authored-by: Olivier Wilkinson (reivilibre) <oliverw@matrix.org>
2024-04-09 14:07:26 +01:00
Mathieu Velten
e363881592
Fix PR #16677, a parameter was missing in a function call (#17033)
Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-04-09 14:06:46 +01:00
Erik Johnston
d40878451c
Add forgotten schema delta (#17054)
This should have been in #17045. Whoops.
2024-04-09 13:03:41 +01:00
Erik Johnston
4d10a8fb18
Fixups to #17064 (#17065)
Forget a line, and an empty batch is trivially linear.

c.f. #17064
2024-04-08 14:55:19 +01:00
Erik Johnston
1f8f991d51
Add back fast path for non-gappy syncs (#17064)
PR #16942 removed an invalid optimisation that avoided pulling out state
for non-gappy syncs. This causes a large increase in DB usage. c.f.
#16941 for why that optimisation was wrong.

However, we can still optimise in the simple case where the events in
the timeline are a linear chain without any branching/merging of the
DAG.

cc. @richvdh
2024-04-08 14:25:28 +01:00
Erik Johnston
5360baeb64
Pull out fewer receipts from DB when doing push (#17049)
Before we were pulling out *all* read receipts for a user for every
event we pushed. Instead let's only pull out the relevant receipts.

This also pulled out the event rows for each receipt, causing load on
the events table.
2024-04-05 12:46:34 +01:00
Richard van der Hoff
0e68e9b7f4
Fix bug in calculating state for non-gappy syncs (#16942)
Unfortunately, the optimisation we applied here for non-gappy syncs is
not actually valid.

Fixes https://github.com/element-hq/synapse/issues/16941.

~~Based on https://github.com/element-hq/synapse/pull/16930.~~
Requires https://github.com/matrix-org/sytest/pull/1374.
2024-04-04 16:15:35 +00:00