Commit Graph

18595 Commits

Author SHA1 Message Date
Erik Johnston
14b5b48a22
Fix ratelimiting for federation /send requests. (#8342)
c.f. #8295 for rationale
2020-09-18 10:49:29 +01:00
Jonathan de Jong
efb6b6629c
Move lint dependencies to extras_require (#8330)
Lint dependencies can now be installed with pip install -e ".[lint]"

This should help keep the version in sync between tox and documentation.
2020-09-17 16:45:22 -04:00
Matthew Hodgson
ad055ea4cc blacklist MSC2753 sytests until it's implemented in synapse (#8285)
Dendrite's implementing MSC2753 over at https://github.com/matrix-org/dendrite/pull/1370 to prove the implementation for MSC purposes, and so sytest has sprouted tests for it over at https://github.com/matrix-org/sytest/pull/944. But we don't want them to run on synapse until synapse implements it.
2020-09-17 14:02:20 -04:00
Jonathan de Jong
837293c314
Remove obsolete __future__ imports (#8337) 2020-09-17 08:37:01 -04:00
Patrick Cloke
c3c9732c53
Use admin_patterns for all admin APIs. (#8331)
This reduces duplication of the admin prefix in regular expressions.
2020-09-17 07:04:15 -04:00
Jonathan de Jong
53284c425e
Fix a potential bug of UnboundLocalError (#8329)
Replaced with less buggier control flow
2020-09-17 11:54:56 +01:00
Jonathan de Jong
a3f124b821
Switch metaclass initialization to python 3-compatible syntax (#8326) 2020-09-16 15:15:55 -04:00
Patrick Cloke
44dec6cbc4 Synapse 1.20.0rc4 (2020-09-16)
==============================
 
 Synapse 1.20.0rc4 is identical to 1.20.0rc3, with the addition of the security fix that was included in 1.19.2.
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9iC2QACgkQM/xY9qcR
 MEhQJw/+N158AAvxLQsk4bNu8gscc6CAJaDtdR+QNqQa9UOit8dj7nnIhgRCqvPy
 ZyNjKpI0LxytTS+NAYWNT88gfOcODR9TkWd7o8rGLCBenF3SwdZ2MM40gmHkola1
 WunsRSHuYH1lyVumdOOIw8uOOnglORgRjcuQ4ht5AKYYnWAMBxD+crluXOGdJeLS
 OdxBzCCHO1s3nb09ybvpxsxtXwbsRqhDKqotZn3gJzPbtV8NUNaer29bA7bAfMJ0
 NvIKIEjrg79PEYX2G0i2kGaNqcCne8SY8Mb/nc0/M2KiLPDevy+eKA2jKYmWfUK+
 2MbGYmA5rXUy4RPUB4cmGkqbsZhBPMGT4O6GD8UTJ/9dESUPzTJKcJgv1mZbMA1F
 HfQOnjTRrsf9incU+QBgkKkrVboBTnpewO2mRoiLtavAUiPiXX0WvWrYfRpXyiSU
 uihuVHY/tefxEW2uUGYGsCxthm8V2vrIumd8qASdybAO7gZtyrKN/5Z9M1Xhu9Fc
 FidoOXo1TrAjoqVL2rfslPYdl0Pqed4IoWthwSIV+0C33e3qzeH32X2unY0HX4s/
 3KPvxXqAsdE6T0WdZwKEDrGO33OtrBPkJK+O2yjSV+3d+wdqGm+RVm39lsVysKX2
 KJXcJ/An8yDwaN+lemeSLbOxU3TRWGdT/Wrek8HAH/bAULzlLWg=
 =0feV
 -----END PGP SIGNATURE-----

Merge tag 'v1.20.0rc4' into develop

Synapse 1.20.0rc4 (2020-09-16)
==============================

Synapse 1.20.0rc4 is identical to 1.20.0rc3, with the addition of the security fix that was included in 1.19.2.
2020-09-16 09:00:20 -04:00
Patrick Cloke
7141057e85 1.20.0rc4 2020-09-16 08:54:30 -04:00
Patrick Cloke
ab165994db Merge remote-tracking branch 'origin/master' into release-v1.20.0 2020-09-16 08:52:21 -04:00
Erik Johnston
7c43643037 Merge branch 'master' into develop 2020-09-16 13:50:10 +01:00
Erik Johnston
5ffd68dca1 1.19.2 2020-09-16 13:37:03 +01:00
Erik Johnston
5dd051acae Merge branch 'erikj/fix_origin_check' into develop 2020-09-16 12:43:02 +01:00
Erik Johnston
9c8ef134fe Merge branch 'erikj/fix_origin_check' into release-v1.20.0 2020-09-16 12:42:42 +01:00
Erik Johnston
f1c9ded738 Merge branch 'erikj/fix_origin_check' into release-v1.19.2 2020-09-16 12:40:58 +01:00
Erik Johnston
97659b7489 Newsfile 2020-09-16 12:05:01 +01:00
Erik Johnston
c570f24acc Don't assume that an event has an origin field
This fixes #8319.
2020-09-16 11:56:23 +01:00
reivilibre
576bc37d31
Catch-up after Federation Outage (split, 4): catch-up loop (#8272) 2020-09-15 09:07:19 +01:00
Patrick Cloke
aec294ee0d
Use slots in attrs classes where possible (#8296)
slots use less memory (and attribute access is faster) while slightly
limiting the flexibility of the class attributes. This focuses on objects
which are instantiated "often" and for short periods of time.
2020-09-14 12:50:06 -04:00
Patrick Cloke
d2a3eb04a4 Fix typos in comments. 2020-09-14 11:46:58 -04:00
Tulir Asokan
b82d68c0bd
Add the topic and avatar to the room details admin API (#8305) 2020-09-14 10:07:04 -04:00
Patrick Cloke
6605470bfb
Improve SAML error messages (#8248) 2020-09-14 09:05:36 -04:00
Erik Johnston
04cc249b43
Add experimental support for sharding event persister. Again. (#8294)
This is *not* ready for production yet. Caveats:

1. We should write some tests...
2. The stream token that we use for events can get stalled at the minimum position of all writers. This means that new events may not be processed and e.g. sent down sync streams if a writer isn't writing or is slow.
2020-09-14 10:16:41 +01:00
Patrick Cloke
a9dbe98ef9 Synapse 1.20.0rc3 (2020-09-11)
==============================
 
 Bugfixes
 --------
 
 - Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
 -----BEGIN PGP SIGNATURE-----
 
 iQIzBAABCAAdFiEEF3tZXk38tRDFVnUIM/xY9qcRMEgFAl9bbA8ACgkQM/xY9qcR
 MEjjJg//ZGcvPLr8y3F5JXptcjWA3AFja3DgztBA3uNFoDrwVhR9m3+F4rL/Fbat
 gnaccWsKIXOjZw5WOThQEUIfYMKmHEsaDl7xtlUPC4ylmSV+31ypu1KiRSxND2kT
 TmOupvqw+4/E4RajpYX6WT3e2oQUpIcBAKsUyZL3ZXECR/yNYrA6w2w7i4wrv3cU
 QDzBFrCcr5aJql7VUc88BlaZUG6xom2/kkPtjmO6imPnPGBHLN22uPU7zQ4n7Vgi
 Nrg45v5WHVCx57WoXqAEap1zdKgoRCna1x0NkqYh0OAXq1l6aVwlf/Pdynt91f2x
 yyaMYjMskjyatHlzONMV4kz0w03dUrGJXiAx2ldEDd32SKBUxLJ/CYtNEdoXZ4Mm
 o0OJfDbaSJwlqK7FhdZJE3kCSkUvlmVasDErXfQRDVHy5sx9Ufr4P0kkaZyXDys1
 UAWFcCJwy9dgFR2679PoXk1s/gHf10wuk6FOs8ESbcJJOvCljxWrOIqKNzSUL3HQ
 Hj5V1zHYXB6HtgiI7tbKxSn/d2uajHI9b8LVOkyV9RKsb1DGGRgNQQ5Kz4Zud7dC
 vuQ2jVw7JIo41W3QIXvcL1KdQeje2nwYwuwNREZaEDZJsGLjxaJ6dNyBVgkmYYFP
 gxhplSOSsFjQRGEsBrc2utWxDnLVGPHEAt7iHVSqjwVUyARvxjo=
 =e2T1
 -----END PGP SIGNATURE-----

Merge tag 'v1.20.0rc3' into develop

Synapse 1.20.0rc3 (2020-09-11)
==============================

Bugfixes
--------

- Fix a bug introduced in v1.20.0rc1 where the wrong exception was raised when invalid JSON data is encountered. ([\#8291](https://github.com/matrix-org/synapse/issues/8291))
2020-09-11 08:30:36 -04:00
Patrick Cloke
08837bb58c Clarify changelog. 2020-09-11 08:21:57 -04:00
Patrick Cloke
2832ef5bb7 1.20.0rc3 2020-09-11 08:14:15 -04:00
Erik Johnston
fe8ed1b46f
Make StreamToken.room_key be a RoomStreamToken instance. (#8281) 2020-09-11 12:22:55 +01:00
Patrick Cloke
b86764662b
Fix the exception that is raised when invalid JSON is encountered. (#8291) 2020-09-10 14:55:25 -04:00
Dan Callaghan
c312ee3cde
Use TLSv1.2 for fake servers in tests (#8208)
Some Linux distros have begun disabling TLSv1.0 and TLSv1.1 by default
for security reasons, for example in Fedora 33 onwards:

https://fedoraproject.org/wiki/Changes/StrongCryptoSettings2

Use TLSv1.2 for the fake TLS servers created in the test suite, to avoid
failures due to OpenSSL disallowing TLSv1.0:

    <twisted.python.failure.Failure OpenSSL.SSL.Error: [('SSL routines',
    'ssl_choose_client_version', 'unsupported protocol')]>

Signed-off-by: Dan Callaghan <djc@djc.id.au>
2020-09-10 19:49:08 +01:00
Andrew Morgan
95d869c357
Add /_synapse/client to the reverse proxy docs (#8227)
This PR adds a information about forwarding `/_synapse/client` endpoints through your reverse proxy. The first of these endpoints are introduced in https://github.com/matrix-org/synapse/pull/8004.
2020-09-10 13:26:34 +01:00
Erik Johnston
5d3e306d9f
Clean up Notifier.on_new_room_event code path (#8288)
The idea here is that we pass the `max_stream_id` to everything, and only use the stream ID of the particular event to figure out *when* the max stream position has caught up to the event and we can notify people about it.

This is to maintain the distinction between the position of an item in the stream (i.e. event A has stream ID 513) and a token that can be used to partition the stream (i.e. give me all events after stream ID 352). This distinction becomes important when the tokens are more complicated than a single number, which they will be once we start tracking the position of multiple writers in the tokens.

The valid operations here are:

1. Is a position before or after a token
2. Fetching all events between two tokens
3. Merging multiple tokens to get the "max", i.e. `C = max(A, B)` means that for all positions P where P is before A *or* before B, then P is before C.

Future PR will change the token type to a dedicated type.
2020-09-10 13:24:43 +01:00
Andrew Morgan
192e98111d
Remove shared rooms info from upgrade/workers doc as it's still experimental (#8290) 2020-09-10 13:08:08 +01:00
Andrew Morgan
a3a90ee031
Show a confirmation page during user password reset (#8004)
This PR adds a confirmation step to resetting your user password between clicking the link in your email and your password actually being reset.

This is to better align our password reset flow with the industry standard of requiring a confirmation from the user after email validation.
2020-09-10 11:45:12 +01:00
Richard van der Hoff
e44e9ee518 Merge branch 'release-v1.20.0' into develop 2020-09-10 10:15:02 +01:00
Patrick Cloke
b312769c0e
Do not error when thumbnailing invalid files (#8236)
If a file cannot be thumbnailed for some reason (e.g. the file is empty), then
catch the exception and convert it to a reasonable error message for the client.
2020-09-09 12:59:41 -04:00
Patrick Cloke
2ea1c68249
Remove some unused distributor signals (#8216)
Removes the `user_joined_room` and stops calling it since there are no observers.

Also cleans-up some other unused signals and related code.
2020-09-09 12:22:00 -04:00
Richard van der Hoff
536f4a2482 1.20.0rc2 2020-09-09 17:08:33 +01:00
Erik Johnston
c9dbee50ae
Fixup pusher pool notifications (#8287)
`pusher_pool.on_new_notifications` expected a min and max stream ID, however that was not what we were passing in. Instead, let's just pass it the current max stream ID and have it track the last stream ID it got passed.

I believe that it mostly worked as we called the function for every event. However, it would break for events that got persisted out of order, i.e, that were persisted but the max stream ID wasn't incremented as not all preceding events had finished persisting, and push for that event would be delayed until another event got pushed to the effected users.
2020-09-09 16:56:08 +01:00
Erik Johnston
dc9dcdbd59 Revert "Fixup pusher pool notifications"
This reverts commit e7fd336a53.
2020-09-09 16:19:22 +01:00
Erik Johnston
e7fd336a53 Fixup pusher pool notifications 2020-09-09 16:17:50 +01:00
Matthew Hodgson
453dfe210b
blacklist MSC2753 sytests until it's implemented in synapse (#8285)
Dendrite's implementing MSC2753 over at https://github.com/matrix-org/dendrite/pull/1370 to prove the implementation for MSC purposes, and so sytest has sprouted tests for it over at https://github.com/matrix-org/sytest/pull/944. But we don't want them to run on synapse until synapse implements it.
2020-09-09 13:25:59 +01:00
reivilibre
a5370072b5
Don't remember enabled of deleted push rules and properly return 404 for missing push rules in .../actions and .../enabled (#7796)
Signed-off-by: Olivier Wilkinson (reivilibre) <olivier@librepush.net>

Co-authored-by: Richard van der Hoff <1389908+richvdh@users.noreply.github.com>
2020-09-09 11:39:39 +01:00
Patrick Cloke
e45b834119
Add types to async_helpers (#8260) 2020-09-08 16:50:51 -04:00
Erik Johnston
1553adc831
Fix mypy error on develop (#8282) 2020-09-08 17:43:31 +01:00
DeepBlueV7.X
560f3b8609
Include method in thumbnail media name (#7124)
This fixes an issue where different methods (crop/scale) overwrite each other.

This first tries the new path. If that fails and we are looking for a
remote thumbnail, it tries the old path. If that still isn't found, it
continues as normal.

This should probably be removed in the future, after some of the newer
thumbnails were generated with the new path on most deployments. Then
the overhead should be minimal if the other thumbnails need to be
regenerated.

Signed-off-by: Nicolas Werner <nicolas.werner@hotmail.de>
2020-09-08 17:19:50 +01:00
Erik Johnston
63c0e9e195
Add types to StreamToken and RoomStreamToken (#8279)
The intention here is to change `StreamToken.room_key` to be a `RoomStreamToken` in a future PR, but that is a big enough change without this refactoring too.
2020-09-08 16:48:15 +01:00
Andrew Morgan
094896a69d
Add a config option for validating 'next_link' parameters against a domain whitelist (#8275)
This is a config option ported over from DINUM's Sydent: https://github.com/matrix-org/sydent/pull/285

They've switched to validating 3PIDs via Synapse rather than Sydent, and would like to retain this functionality.

This original purpose for this change is phishing prevention. This solution could also potentially be replaced by a similar one to https://github.com/matrix-org/synapse/pull/8004, but across all `*/submit_token` endpoint.

This option may still be useful to enterprise even with that safeguard in place though, if they want to be absolutely sure that their employees don't follow links to other domains.
2020-09-08 16:03:09 +01:00
Brendan Abolivier
d4daff9b59
Fix /notifications and pushers misbehaving because of unread counts (#8280) 2020-09-08 15:26:06 +01:00
Erik Johnston
0f545e6b96
Clean up types for PaginationConfig (#8250)
This removes `SourcePaginationConfig` and `get_pagination_rows`. The reasoning behind this is that these generic classes/functions erased the types of the IDs it used (i.e. instead of passing around `StreamToken` it'd pass in e.g. `token.room_key`, which don't have uniform types).
2020-09-08 15:00:17 +01:00
Richard van der Hoff
703e2b8a96
Use the right constructor for log records (#8278)
Update `log_function` to use the right factory to create log records, to make
sure that they have `request` attributes.

Fixes: #8267.
2020-09-08 14:52:51 +01:00