Commit Graph

1747 Commits

Author SHA1 Message Date
Erik Johnston
74b74462f1
Fix /events/:event_id deprecated API. (#6731) 2020-01-20 17:38:09 +00:00
Erik Johnston
ceecedc68b
Fix changing password via user admin API. (#6730) 2020-01-20 17:23:59 +00:00
Brendan Abolivier
3b72bb780a
Merge pull request #6714 from matrix-org/babolivier/retention_select_event
Fix instantiation of message retention purge jobs
2020-01-17 14:23:51 +00:00
Brendan Abolivier
842c2cfbf1
Remove get_room_event_after_stream_ordering entirely 2020-01-16 20:24:17 +00:00
Neil Johnson
38e0e59f42
Add org.matrix.e2e_cross_signing to unstable_features in /versions as per MSC1756 (#6712) 2020-01-16 09:46:14 +00:00
Andrew Morgan
edc244eec4
Remove duplicate session check in web fallback servlet (#6702) 2020-01-15 18:05:18 +00:00
Andrew Morgan
1177d3f3a3
Quarantine media by ID or user ID (#6681) 2020-01-13 18:10:43 +00:00
Manuel Stahl
d2906fe666 Allow admin users to create or modify users without a shared secret (#6495)
Signed-off-by: Manuel Stahl <manuel.stahl@awesome-technologies.de>
2020-01-09 13:31:00 +00:00
Richard van der Hoff
573fee759c
Back out ill-advised notary server hackery (#6657)
This was ill-advised. We can't modify verify_keys here, because the response
object has already been signed by the requested key.

Furthermore, it's somewhat unnecessary because existing versions of Synapse
(which get upset that the notary key isn't present in verify_keys) will fall
back to a direct fetch via `/key/v2/server`.

Also: more tests for fetching keys via perspectives: it would be nice if we actually tested when our fetcher can't talk to our notary impl.
2020-01-08 13:24:10 +00:00
Richard van der Hoff
18674eebb1
Workaround for error when fetching notary's own key (#6620)
* Kill off redundant SynapseRequestFactory

We already get the Site via the Channel, so there's no need for a dedicated
RequestFactory: we can just use the right constructor.

* Workaround for error when fetching notary's own key

As a notary server, when we return our own keys, include all of our signing
keys in verify_keys.

This is a workaround for #6596.
2020-01-06 12:28:58 +00:00
Richard van der Hoff
98247c4a0e
Remove unused, undocumented "content repo" resource (#6628)
This looks like it got half-killed back in #888.

Fixes #6567.
2020-01-03 17:10:52 +00:00
Richard van der Hoff
d6752ce5da
Clean up startup for the pusher (#6558)
* Remove redundant python2 support code

`str.decode()` doesn't exist on python3, so presumably this code was doing
nothing

* Filter out pushers with corrupt data

When we get a row with unparsable json, drop the row, rather than returning a
row with null `data`, which will then cause an explosion later on.

* Improve logging when we can't start a pusher

Log the ID to help us understand the problem

* Make email pusher setup more robust

We know we'll have a `data` member, since that comes from the database. What we
*don't* know is if that is a dict, and if that has a `brand` member, and if
that member is a string.
2019-12-18 14:26:58 +00:00
Erik Johnston
72acca6a32 Back out change preventing setting null avatar URLs 2019-12-11 11:46:55 +00:00
Andrew Morgan
5e8abe9013
Better errors regarding changing avatar_url (#6497) 2019-12-09 14:54:33 +00:00
Erik Johnston
2ace775d88 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/make_database_class 2019-12-06 11:33:34 +00:00
Manuel Stahl
649b6bc088 Replace /admin/v1/users_paginate endpoint with /admin/v2/users (#5925) 2019-12-05 18:12:23 +00:00
Erik Johnston
9c41ba4c5f Port rest.client.v2 2019-12-05 16:56:23 +00:00
Erik Johnston
1a0997bbd5 Port rest/v1 to async/await 2019-12-05 15:57:28 +00:00
Erik Johnston
4a33a6dd19 Move background update handling out of store 2019-12-05 11:11:26 +00:00
Brendan Abolivier
9dc84b7989
Merge branch 'develop' into babolivier/context_filters 2019-12-04 14:23:44 +00:00
Filip Štědronský
81731c6e75 Fix: Pillow error when uploading RGBA image (#3325) (#6241)
Signed-Off-By: Filip Štědronský <g@regnarg.cz>
2019-12-02 12:12:55 +00:00
Andrew Morgan
23ea572125
Add User-Interactive Auth to /account/3pid/add (#6119) 2019-11-29 13:51:14 +00:00
Erik Johnston
69d8fb83c6 MSC2367 Allow reason field on all member events 2019-11-28 11:02:04 +00:00
Hubert Chathi
0d27aba900
add etag and count to key backup endpoints (#5858) 2019-11-27 16:14:44 -05:00
Richard van der Hoff
ef1a85e773
Fix startup error when http proxy is defined. (#6421)
Guess I only tested this on python 2 :/

Fixes #6419.
2019-11-26 18:10:50 +00:00
Andrew Morgan
3916e1b97a
Clean up newline quote marks around the codebase (#6362) 2019-11-21 12:00:14 +00:00
Manuel Stahl
4f5ca455bf Move admin endpoints into separate files (#6308) 2019-11-20 11:49:11 +00:00
Brendan Abolivier
83446a18fb
Merge pull request #6335 from matrix-org/erikj/rc_login_cleanups
Only do `rc_login` ratelimiting on succesful login.
2019-11-20 09:52:38 +00:00
Brendan Abolivier
271c322d08
Lint 2019-11-20 09:29:48 +00:00
Erik Johnston
c7376cdfe3
Apply suggestions from code review
Co-Authored-By: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
2019-11-18 17:10:16 +00:00
Erik Johnston
541f1b92d9 Only do rc_login ratelimiting on succesful login.
We were doing this in a number of places which meant that some login
code paths incremented the counter multiple times.

It was also applying ratelimiting to UIA endpoints, which was probably
not intentional.

In particular, some custom auth modules were calling
`check_user_exists`, which incremented the counters, meaning that people
would fail to login sometimes.
2019-11-06 11:08:58 +00:00
Richard van der Hoff
5570d1c93f
Merge pull request #6334 from matrix-org/rav/url_preview_limit_title_2
Fix exception when OpenGraph tag values are ints
2019-11-05 17:28:11 +00:00
Richard van der Hoff
81d49cbb07 Fix exception when OpenGraph tag values are ints 2019-11-05 17:22:58 +00:00
Richard van der Hoff
55a7da247a
Merge branch 'develop' into rav/url_preview_limit_title 2019-11-05 17:08:07 +00:00
Richard van der Hoff
e78167c94b
Apply suggestions from code review
Co-Authored-By: Brendan Abolivier <babolivier@matrix.org>
Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-11-05 16:46:39 +00:00
Richard van der Hoff
e9bfe719ba Strip overlong OpenGraph data from url preview
... to stop people causing DoSes with malicious web pages
2019-11-05 15:51:18 +00:00
Brendan Abolivier
8822b33111
Update copyrights 2019-11-05 14:46:35 +00:00
Brendan Abolivier
f496d25877
Merge pull request #6301 from matrix-org/babolivier/msc2326
Implement MSC2326 (label based filtering)
2019-11-01 17:04:45 +00:00
Richard van der Hoff
1cb84c6486
Support for routing outbound HTTP requests via a proxy (#6239)
The `http_proxy` and `HTTPS_PROXY` env vars can be set to a `host[:port]` value which should point to a proxy.

The address of the proxy should be excluded from IP blacklists such as the `url_preview_ip_range_blacklist`.

The proxy will then be used for
 * push
 * url previews
 * phone-home stats
 * recaptcha validation
 * CAS auth validation

It will *not* be used for:
 * Application Services
 * Identity servers
 * Outbound federation
 * In worker configurations, connections from workers to masters

Fixes #4198.
2019-11-01 14:07:44 +00:00
Brendan Abolivier
e3689ac6f7
Add unstable feature flag 2019-11-01 10:41:23 +00:00
Amber Brown
020add5099
Update black to 19.10b0 (#6304)
* update version of black and also fix the mypy config being overridden
2019-11-01 02:43:24 +11:00
Andrew Morgan
54fef094b3
Remove usage of deprecated logger.warn method from codebase (#6271)
Replace every instance of `logger.warn` with `logger.warning` as the former is deprecated.
2019-10-31 10:23:24 +00:00
Erik Johnston
770d1ef673
Merge pull request #6280 from matrix-org/erikj/receipts_async_await
Port receipt and read markers to async/wait
2019-10-30 11:44:18 +01:00
Erik Johnston
2c35ffead2 Port receipt and read markers to async/wait 2019-10-29 15:08:22 +00:00
Erik Johnston
9be41bc121 Port room rest handlers to async/await 2019-10-29 13:09:29 +00:00
Michael Kaye
e4d98188da Address codestyle concerns 2019-10-24 18:43:13 +01:00
Michael Kaye
8f4a808d9d Delay printf until logging is required.
Using % will cause the string to be generated even if debugging
is off.
2019-10-24 18:31:53 +01:00
Hubert Chathi
36adfaedab Merge branch 'develop' into cross-signing_sig_upload 2019-10-18 18:34:42 +01:00
Hubert Chathi
cfc28325a6 Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-10-18 11:39:40 +01:00
Erik Johnston
d98029ea89
Merge pull request #6196 from matrix-org/erikj/await
Move rest/admin to use async/await.
2019-10-18 11:53:02 +02:00
Erik Johnston
2e97a4c197 Port synapse/rest/client/_base.py to async/await 2019-10-11 15:29:26 +01:00
Erik Johnston
3c2d6c708c Add maybe_awaitable and fix __init__ bugs 2019-10-11 15:26:09 +01:00
Hubert Chathi
691dd67fcd
Merge pull request #6189 from matrix-org/uhoreg/e2e_backup_optional_version
make version optional in body of e2e backup version update
2019-10-11 10:11:59 -04:00
Valérian Rousset
be9b55e0d2 cas: support setting display name (#6114)
Now, the CAS server can return an attribute stating what's the desired displayname, instead of using the username directly.
2019-10-11 12:33:12 +01:00
Erik Johnston
dfbb62c28d Port synaps/rest/client/users.py to async/await 2019-10-11 12:21:40 +01:00
Erik Johnston
f95325e22a Port synaps/rest/client/server_notice_servlet.py to async/await 2019-10-11 12:21:40 +01:00
Erik Johnston
281f887090 Port synaps/rest/client/media.py to async/await 2019-10-11 12:21:40 +01:00
Erik Johnston
fca3a541e7 Port rest/admin/__init__.py to async/await 2019-10-11 12:17:52 +01:00
Erik Johnston
ca3e01e50d Fix store_url_cache using bytes 2019-10-10 14:52:29 +01:00
krombel
2efd050c9d send 404 as http-status when filter-id is unknown to the server (#2380)
This fixed the weirdness of 400 vs 404 as http status code in the case
the filter id is not known by the server.
As e.g. matrix-js-sdk expects 404 to catch this situation this leads
to unwanted behaviour.
2019-10-10 12:59:55 +01:00
Hubert Chathi
4535a07f4a make version optional in body of e2e backup version update
to agree with latest version of the MSC
2019-10-09 17:54:03 -04:00
Anshul Angaria
474abf1eb6 add M_TOO_LARGE error code for uploading a too large file (#6151)
Fixes #6109
2019-10-08 13:55:16 +01:00
Michael Kaye
dc795ba709 Log responder we are using. (#6139)
This prevents us logging "Responding to media request with responder %s".
2019-10-07 15:41:25 +01:00
Robert Swain
39b40d6d99 media/thumbnailer: Better quality for 1-bit / 8-bit color palette images (#2142)
Pillow will use nearest neighbour as the resampling algorithm if the
source image is either 1-bit or a color palette using 8 bits. If we
convert to RGB before scaling, we'll probably get a better result.
2019-10-04 09:34:52 +01:00
Erik Johnston
a5166e4d5f
Land improved room list based on room stats (#6019)
Use room_stats and room_state for room directory search
2019-10-02 14:08:35 +01:00
Erik Johnston
67ee18daea
Merge pull request #6108 from matrix-org/erikj/remove_get_user_by_req-span
Add some helpful opentracing tags and remove get_user_by_req span
2019-09-27 10:02:57 +01:00
Richard van der Hoff
54569c787b
Kill off half-implemented password-reset via sms (#6101)
Doing a password reset via SMS has never worked, and in any case is a silly
idea because msisdn recycling is a thing.

See also matrix-org/matrix-doc#2303.
2019-09-26 15:38:25 +01:00
Richard van der Hoff
990928abde
Stop advertising unsupported flows for registration (#6107)
If email or msisdn verification aren't supported, let's stop advertising them
for registration.

Fixes #6100.
2019-09-25 12:10:26 +01:00
Erik Johnston
dc2c97e1a3 isort 2019-09-25 11:59:05 +01:00
Erik Johnston
5d99713854 Add tags for event_id and txn_id in event sending
This will make it easier to search for sending event requests.
2019-09-25 11:39:15 +01:00
Richard van der Hoff
8004d6ca2f
Refactor code for calculating registration flows (#6106)
because, frankly, it looked like it was written by an axe-murderer.

This should be a non-functional change, except that where `m.login.dummy` was
previously advertised *before* `m.login.terms`, it will now be advertised
afterwards. AFAICT that should have no effect, and will be more consistent with
the flows that involve passing a 3pid.
2019-09-25 11:32:05 +01:00
Richard van der Hoff
ed8b92f0d2 Merge remote-tracking branch 'origin/develop' into rav/saml_mapping_work 2019-09-24 12:57:32 +01:00
Andrew Morgan
2b071a2ff1 Add an unstable feature flag for separate add/bind 3pid APIs (#6044)
Add a m.separate_add_and_bind flag set to True. See MSC2290's Backward Compatibility section for details.
2019-09-23 20:46:34 +01:00
Andrew Morgan
1ea3ed7620 Add m.id_access_token to /versions unstable_features (MSC2264) (#5974)
Adds a flag to /versions' unstable_features section indicating that this Synapse understands what an id_access_token is, as per MSC2264.

Fixes #5927
2019-09-23 20:19:25 +01:00
Andrew Morgan
b38aa82b83 Add m.require_identity_server to /versions unstable_flags (#5972)
As MSC2263 states, m.require_identity_server must be set to false when it does not require an identity server to be provided by the client for the purposes of email registration or password reset.

Adds an m.require_identity_server flag to /versionss unstable_flags section. This will advertise that Synapse no longer needs id_server as a parameter.
2019-09-23 18:52:43 +01:00
Andrew Morgan
2c99c63453 Add POST submit_token endpoint for MSISDN (#6078)
First part of solving #6076
2019-09-23 17:49:00 +01:00
Andrew Morgan
30af161af2 Implement MSC2290 (#6043)
Implements MSC2290. This PR adds two new endpoints, /unstable/account/3pid/add and /unstable/account/3pid/bind. Depending on the progress of that MSC the unstable prefix may go away.

This PR also removes the blacklist on some 3PID tests which occurs in #6042, as the corresponding Sytest PR changes them to use the new endpoints.

Finally, it also modifies the account deactivation code such that it doesn't just try to deactivate 3PIDs that were bound to the user's account, but any 3PIDs that were bound through the homeserver on that user's account.
2019-09-23 16:50:27 +01:00
Andrew Morgan
1b519e0272 Disable /register/available if registration is disabled (#6082)
Fixes #6066

This register endpoint should be disabled if registration is disabled, otherwise we're giving anyone the ability to check if a username exists on a server when we don't need to be.

Error code is 403 (Forbidden) as that's the same returned by /register when registration is disabled.
2019-09-23 15:38:38 +01:00
Andrew Morgan
df3401a71d
Allow HS to send emails when adding an email to the HS (#6042) 2019-09-20 15:21:30 +01:00
Andrew Morgan
7763dd3e95
Remove trailing slash ability from password reset's submit_token endpoint (#6074)
Remove trailing slash ability from the password reset submit_token endpoint. Since we provide the link in an email, and have never sent it with a trailing slash, there's no point for us to accept them on the endpoint.
2019-09-20 14:58:37 +01:00
Andrew Morgan
3ac614eb6c
Drop support for bind param on POST /account/3pid (MSC2290) (#6067)
As per [MSC2290](https://github.com/matrix-org/matrix-doc/pull/2290/files#diff-05cde9463e9209b701312b3baf2fb2ebR151), we're dropping the bind parameter from `/account/3pid`. This endpoint can now only be used for adding threepid's to the user's account on the homeserver.
2019-09-20 10:46:34 +01:00
J. Ryan Stinnett
36015d68ef Use unstable prefix for 3PID unbind API (#6062) 2019-09-19 22:28:29 +01:00
Richard van der Hoff
b65327ff66 Merge branch 'develop' into rav/saml_mapping_work 2019-09-19 18:13:31 +01:00
Andrew Morgan
62e3ff92fd
Remove POST method from password reset submit_token endpoint (#6056)
Removes the POST method from `/password_reset/<medium>/submit_token/` as it's only used by phone number verification which Synapse does not support yet.
2019-09-19 10:53:14 +01:00
Richard van der Hoff
a8ac40445c Record mappings from saml users in an external table
We want to assign unique mxids to saml users based on an incrementing
suffix. For that to work, we need to record the allocated mxid in a separate
table.
2019-09-13 16:01:46 +01:00
Andrew Morgan
9fc71dc5ee
Use the v2 Identity Service API for lookups (MSC2134 + MSC2140) (#5976)
This is a redo of https://github.com/matrix-org/synapse/pull/5897 but with `id_access_token` accepted.

Implements [MSC2134](https://github.com/matrix-org/matrix-doc/pull/2134) plus Identity Service v2 authentication ala [MSC2140](https://github.com/matrix-org/matrix-doc/pull/2140).

Identity lookup-related functions were also moved from `RoomMemberHandler` to `IdentityHandler`.
2019-09-11 16:02:42 +01:00
Andrew Morgan
b5833a2abf Add changelog 2019-09-10 17:56:10 +01:00
Andrew Morgan
60d3c57bd0 Use account_threepid_delegate for 3pid validation 2019-09-10 17:56:10 +01:00
Hubert Chathi
8e86f5b65c Merge branch 'develop' into uhoreg/e2e_cross-signing_merged 2019-09-07 13:20:34 -04:00
Hubert Chathi
0d61d1d735 Merge branch 'develop' into cross-signing_sig_upload 2019-09-07 13:14:45 -04:00
Andrew Morgan
cf5a420c8a
Apply suggestions from code review
Co-Authored-By: Erik Johnston <erik@matrix.org>
2019-09-06 13:34:42 +01:00
Andrew Morgan
ca74b140f2 Fix destructuring assumption bug 2019-09-06 13:25:06 +01:00
Andrew Morgan
0c0b82b6d1
Allow Synapse to send registration emails + choose Synapse or an external server to handle 3pid validation (#5987)
This is a combination of a few different PRs, finally all being merged into `develop`:

* #5875 
* #5876 
* #5868 (This one added the `/versions` flag but the flag itself was actually [backed out](891afb57cb (diff-e591d42d30690ffb79f63bb726200891)) in #5969. What's left is just giving /versions access to the config file, which could be useful in the future)
* #5835 
* #5969 
* #5940

Clients should not actually use the new registration functionality until https://github.com/matrix-org/synapse/pull/5972 is merged.

UPGRADE.rst, changelog entries and config file changes should all be reviewed closely before this PR is merged.
2019-09-06 11:35:28 +01:00
Andrew Morgan
a0d294c306
Switch to using v2 Identity Service APIs other than lookup (MSC 2140) (#5892) 2019-09-05 14:31:22 +01:00
Andrew Morgan
90d17a3d28
Add POST /_matrix/client/r0/account/3pid/unbind (MSC2140) (#5980)
Implements `POST /_matrix/client/r0/account/3pid/unbind` from [MSC2140](https://github.com/matrix-org/matrix-doc/blob/dbkr/tos_2/proposals/2140-terms-of-service-2.md#post-_matrixclientr0account3pidunbind).
2019-09-05 14:00:30 +01:00
Hubert Chathi
c8dc740a94 update with newer coding style 2019-09-04 22:30:45 -04:00
Hubert Chathi
7d6c70fc7a make black happy 2019-09-04 22:04:12 -04:00
Hubert Chathi
ac4746ac4b allow uploading signatures of master key signed by devices 2019-09-04 22:03:41 -04:00
Hubert Chathi
4bb4544784 implement device signature uploading/fetching 2019-09-04 20:02:56 -04:00
Hubert Chathi
faf72a4c40 Merge branch 'develop' into cross-signing_keys 2019-09-04 19:12:29 -04:00