Commit Graph

2373 Commits

Author SHA1 Message Date
Connor Davis
bb5b47b62a
Add Admin API to Fetch Messages Within a Particular Window ()
This adds two new admin APIs that allow us to fetch messages from a room within a particular time.
2022-09-07 10:54:44 +01:00
Richard van der Hoff
e8130f219b
Return keys for unwhitelisted servers from /_matrix/key/v2/query () 2022-09-01 13:54:02 +01:00
Šimon Brandner
0e99f07952
Remove support for unstable private read receipts ()
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-09-01 13:31:54 +01:00
Jacek Kuśnierz
84ddcd7bbf
Drop support for calling /_matrix/client/v3/rooms/{roomId}/invite without an id_access_token ()
Fixes 

Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de
2022-08-31 12:10:25 +00:00
reivilibre
7bc110a19e
Generalise the @cancellable annotation so it can be used on functions other than just servlet methods. () 2022-08-31 11:16:05 +00:00
Erik Johnston
1c26acd815
Fix bug where we wedge media plugins if clients disconnect early ()
We incorrectly didn't use the returned `Responder` if the client had
disconnected, which meant that the resource used by the Responder
wasn't correctly released.

In particular, this exhausted the thread pools so that *all* requests
timed out.
2022-08-30 12:17:48 +01:00
Patrick Cloke
303b40b988
Do not wait for background updates to complete do expire URL cache. ()
Media downloaded as part of a URL preview is normally deleted after two days.
However, while a background database migration is running, the process is
stopped. A long-running database migration can therefore cause the media
store to fill up with old preview files.

This logic was added in  to make sure that we didn't try to run the expiry
without an index on `local_media_repository.created_ts`; the original logic that
needs that index was added in  (in `get_url_cache_media_before`, as
amended by 93247a424a), and is still present.

Given that the background update was added before Synapse v1.0.0, just drop
this check and assume the index exists.
2022-08-30 07:15:54 -04:00
David Robertson
956e015413
Drop support for delegating email validation, round 2 () 2022-08-23 11:40:00 +00:00
Eric Eastwood
9385c41ba4
Fix Prometheus metrics being negative (mixed up start/end) ()
Fix:

 - https://github.com/matrix-org/synapse/pull/13535#discussion_r949582508
 - https://github.com/matrix-org/synapse/pull/13533#discussion_r949577244
2022-08-23 08:47:30 +01:00
Quentin Gliech
3dd175b628
synapse.api.auth.Auth cleanup: make permission-related methods use Requester instead of the UserID ()
Part of 

This changes all the permission-related methods to rely on the Requester instead of the UserID. This is a first step towards enabling scoped access tokens at some point, since I expect the Requester to have scope-related informations in it.

It also changes methods which figure out the user/device/appservice out of the access token to return a Requester instead of something else. This avoids having store-related objects in the methods signatures.
2022-08-22 14:17:59 +01:00
Andrew Morgan
94375f7a91
Remove redundant opentracing spans for /sendToDevice and /keys/upload () 2022-08-22 10:03:11 +01:00
Andrew Morgan
f9f03426de
Implement MSC3852: Expose last_seen_user_agent to users for their own devices; also expose to Admin API () 2022-08-19 16:17:10 +00:00
reivilibre
3a245f6cfe
Fix validation problem that occurs when a user tries to deactivate their account or change their password. () 2022-08-19 11:03:29 +00:00
Eric Eastwood
2c42673a9b
Add metrics to track /messages response time by room size ()
Follow-up to https://github.com/matrix-org/synapse/pull/13533

Part of https://github.com/matrix-org/synapse/issues/13356
2022-08-18 14:15:53 -05:00
Dirk Klimpel
d75512d19e
Add forgotten status to Room Details API () 2022-08-17 09:42:01 +00:00
Eric Eastwood
2c8cfd6d85
Add specific metric to time long-running /messages requests () 2022-08-17 10:19:21 +01:00
David Robertson
d642ce4b32
Use Pydantic to systematically validate a first batch of endpoints in synapse.rest.client.account. () 2022-08-15 19:05:57 +00:00
Šimon Brandner
ab18441573
Support stable identifiers for MSC2285: private read receipts. ()
This adds support for the stable identifiers of MSC2285 while
continuing to support the unstable identifiers behind the configuration
flag. These will be removed in a future version.
2022-08-05 11:09:33 -04:00
Dirk Klimpel
d6e94ad9d9
Rename RateLimitConfig to RatelimitSettings () 2022-08-03 10:40:20 +01:00
Richard van der Hoff
d548d8f18d Synapse 1.64.0rc2 (2022-07-29)
==============================
 
 This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
 -----BEGIN PGP SIGNATURE-----
 
 iQEzBAABCgAdFiEEv27Axt/F4vrTL/8QOSor00I9eP8FAmLj4a8ACgkQOSor00I9
 eP8biQf/c8yY2mbeRZcBKtp6yoQCRYQvboSMEXyi+dLe1hNqdhSZwRQcAoFuAFwE
 WdScDvoTaElUxv0v6eCI1x9CoXnZ6xpDShvK39j5Yhzv+1tNsm5Uq9imyG3jK5i6
 U/3Gt6CrCsS01VkGslQ3B5I6MFtbC6ZZK9O48yg+GD8Oqw2HH/gllr5swyVbKdbc
 GGhRBHvgXn+w6d/KnKt8uRxJqIpDt9JMga+WdB8CwFR5WnWbGdw24KsyxmBuOLC3
 caQRiluJL/X4jApUpfsJMBBd/jrDod5wWDFO/4P+v0+2d3Ts+hKezZbt5h1VIYSw
 szZXbzxn5RNDkNiJDpOOOMYQ5DXGmA==
 =3/nK
 -----END PGP SIGNATURE-----

Merge tag 'v1.64.0rc2' into develop

Synapse 1.64.0rc2 (2022-07-29)
==============================

This RC reintroduces support for `account_threepid_delegates.email`, which was removed in 1.64.0rc1. It remains deprecated and will be removed altogether in a future release. ([\#13406](https://github.com/matrix-org/synapse/issues/13406))
2022-07-29 15:15:21 +01:00
3nprob
98fb610cc0
Revert "Drop support for delegating email validation ()" ()
Reverts commit fa71bb18b5, and tweaks documentation.

Signed-off-by: 3nprob <git@3n.anonaddy.com>
2022-07-29 10:29:23 +00:00
Šimon Brandner
583f22780f
Use stable prefixes for MSC3827: filtering of /publicRooms by room type ()
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-07-27 19:46:57 +01:00
Patrick Cloke
50122754c8
Add missing types to opentracing. ()
After this change `synapse.logging` is fully typed.
2022-07-21 12:01:52 +00:00
Patrick Cloke
a6895dd576
Add type annotations to trace decorator. ()
Functions that are decorated with `trace` are now properly typed
and the type hints for them are fixed.
2022-07-19 14:14:30 -04:00
Eric Eastwood
7b67e93d49
Provide more info why we don't have any thumbnails to serve ()
Fix https://github.com/matrix-org/synapse/issues/13016

## New error code and status

### Before

Previously, we returned a `404` for `/thumbnail` which isn't even in the spec.

```json
{
  "errcode": "M_NOT_FOUND",
  "error": "Not found [b'hs1', b'tefQeZhmVxoiBfuFQUKRzJxc']"
}
```

### After

What does the spec say?

> 400: The request does not make sense to the server, or the server cannot thumbnail the content. For example, the client requested non-integer dimensions or asked for negatively-sized images.
>
> *-- https://spec.matrix.org/v1.1/client-server-api/#get_matrixmediav3thumbnailservernamemediaid*

Now with this PR, we respond with a `400` when we don't have thumbnails to serve and we explain why we might not have any thumbnails.

```json
{
    "errcode": "M_UNKNOWN",
    "error": "Cannot find any thumbnails for the requested media ([b'example.com', b'12345']). This might mean the media is not a supported_media_format=(image/jpeg, image/jpg, image/webp, image/gif, image/png) or that thumbnailing failed for some other reason. (Dynamic thumbnails are disabled on this server.)",
}
```

> Cannot find any thumbnails for the requested media ([b'example.com', b'12345']). This might mean the media is not a supported_media_format=(image/jpeg, image/jpg, image/webp, image/gif, image/png) or that thumbnailing failed for some other reason. (Dynamic thumbnails are disabled on this server.)


---

We still respond with a 404 in many other places. But we can iterate on those later and maybe keep some in some specific places after spec updates/clarification: https://github.com/matrix-org/matrix-spec/issues/1122

We can also iterate on the bugs where Synapse doesn't thumbnail when it should in other issues/PRs.
2022-07-15 11:42:21 -05:00
Patrick Cloke
1d5c80b161
Reduce duplicate code in receipts servlets. () 2022-07-13 13:23:16 -04:00
Thomas Weston
0312ff44c6
Fix "add user" admin api error when request contains a "msisdn" threepid ()
Co-authored-by: Thomas Weston <thomas.weston@clearspancloud.com>
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-07-13 11:33:21 +01:00
Patrick Cloke
1381563988
Inline URL preview documentation. ()
Inline URL preview documentation near the implementation.
2022-07-12 15:01:58 -04:00
Jacek Kuśnierz
7218a0ca18
Drop support for calling /_matrix/client/v3/account/3pid/bind without an id_access_token ()
Fixes 

Signed-off-by: Jacek Kusnierz jacek.kusnierz@tum.de
2022-07-12 18:48:29 +00:00
Richard van der Hoff
fa71bb18b5
Drop support for delegating email validation ()
* Drop support for delegating email validation

Delegating email validation to an IS is insecure (since it allows the owner of
the IS to do a password reset on your HS), and has long been deprecated. It
will now cause a config error at startup.

* Update unit test which checks for email verification

Give it an `email` config instead of a threepid delegate

* Remove unused method `requestEmailToken`

* Simplify config handling for email verification

Rather than an enum and a boolean, all we need here is a single bool, which
says whether we are or are not doing email verification.

* update docs

* changelog

* upgrade.md: fix typo

* update version number

this will be in 1.64, not 1.63

* update version number

this one too
2022-07-12 19:18:53 +01:00
Quentin Gliech
b19060a29b
Make the AS login method call Auth.get_user_by_req for checking the AS token. ()
This gets rid of another usage of get_appservice_by_req, with all the benefits, including correctly tracking the appservice IP and setting the tracing attributes correctly.

Signed-off-by: Quentin Gliech <quenting@element.io>
2022-07-12 18:06:29 +01:00
David Teller
11f811470f
Uniformize spam-checker API, part 5: expand other spam-checker callbacks to return Tuple[Codes, dict] ()
Signed-off-by: David Teller <davidt@element.io>
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-07-11 16:52:10 +00:00
David Robertson
d70ff5cc35
Extra validation for rest/client/account_data ()
* Extra validation for rest/client/account_data

This is a fairly simple endpoint and we did pretty well here.

* Changelog
2022-07-01 11:04:56 +01:00
Šimon Brandner
13e359aec8
Implement MSC3827: Filtering of /publicRooms by room type ()
Signed-off-by: Šimon Brandner <simon.bra.ag@gmail.com>
2022-06-29 17:12:45 +00:00
Andrew Morgan
bc9b0912cc fix linting error from the 1.61.1 main -> develop merge 2022-06-28 16:47:04 +01:00
Andrew Morgan
6cba6a51af Merge branch 'master' into develop 2022-06-28 15:19:48 +01:00
reivilibre
fa13080618
Merge pull request from GHSA-22p3-qrh9-cx32
* Make _iterate_over_text easier to read by using simple data structures

* Prefer a set of tags to ignore

In my tests, it's 4x faster to check for containment in a set of this size

* Add a stack size limit to _iterate_over_text

* Continue accepting the case where there is no body element

* Use an early return instead for None

Co-authored-by: Richard van der Hoff <richard@matrix.org>
2022-06-28 14:29:08 +01:00
santhoshivan23
6b99a66fe0
Remove unspecced DELETE endpoint that modifies room visibility () 2022-06-28 11:22:59 +00:00
Robert Long
9b683ea80f
Add Cross-Origin-Resource-Policy header to thumbnail and download media endpoints () 2022-06-27 14:44:05 +01:00
santhoshivan23
d54909956e
validate room alias before interacting with the room directory () 2022-06-22 15:32:18 +01:00
Quentin Gliech
73af10f419
Simplify the alias deletion logic as an application service. () 2022-06-17 12:19:22 +01:00
Quentin Gliech
c6d6176411
Allow MSC3030 'timestamp_to_event' calls from anyone on world-readable rooms. ()
Signed-off-by: Quentin Gliech <quenting@element.io>
2022-06-17 11:39:26 +01:00
Patrick Cloke
0fcc0ae37c
Improve URL previews for sites with only Twitter card information. ()
Pull out `twitter:` meta tags when generating a preview and
use it to augment any `og:` meta tags.

Prefers Open Graph information over Twitter card information.
2022-06-16 07:41:57 -04:00
Jacek Kuśnierz
0ef1307619
Add custom well-known ()
Co-authored-by: David Robertson <david.m.robertson1@gmail.com>
2022-06-16 11:48:18 +01:00
Hannes Lerchl
7d99414edf
Replace pyjwt with authlib in org.matrix.login.jwt () 2022-06-15 16:45:16 +00:00
David Robertson
c99b511db9
Fix destination_is errors seen in sentry. ()
* Rename test_fedclient to match its source file
* Require at least one destination to be truthy
* Explicitly validate user ID in profile endpoint GETs
Co-authored-by: Patrick Cloke <clokep@users.noreply.github.com>
2022-06-14 18:28:26 +01:00
Quentin Gliech
fe1daad672
Move the "email unsubscribe" resource, refactor the macaroon generator & simplify the access token verification logic. ()
This simplifies the access token verification logic by removing the `rights`
parameter which was only ever used for the unsubscribe link in email
notifications. The latter has been moved under the `/_synapse` namespace,
since it is not a standard API.

This also makes the email verification link more secure, by embedding the
app_id and pushkey in the macaroon and verifying it. This prevents the user
from tampering the query parameters of that unsubscribe link.

Macaroon generation is refactored:

- Centralised all macaroon generation and verification logic to the
  `MacaroonGenerator`
- Moved to `synapse.utils`
- Changed the constructor to require only a `Clock`, hostname, and a secret key
  (instead of a full `Homeserver`).
- Added tests for all methods.
2022-06-14 09:12:08 -04:00
David Teller
a164a46038
Uniformize spam-checker API, part 4: port other spam-checker callbacks to return Union[Allow, Codes]. ()
Co-authored-by: Brendan Abolivier <babolivier@matrix.org>
2022-06-13 18:16:16 +00:00
Patrick Cloke
dd2d66b0c9
Move the (unstable) dir parameter for /relations behind an experimental flag. ()
MSC3715 defines this parameter, but the unstable version of it should be
behind an experimental flag.
2022-06-08 09:00:35 -04:00
David Robertson
f30bcbd84a
Fix Synapse git info missing in version strings () 2022-06-07 15:24:11 +01:00