Matrix-Mirrors/forked-synapse
1
0
Fork 0
mirror of https://mau.dev/maunium/synapse.git synced 2024-10-01 01:36:05 -04:00
Code Issues Packages Projects Releases Wiki Activity
17,650 Commits 5 Branches 723 Tags 124 MiB
62894673e6
Commit Graph

1880 Commits

Author SHA1 Message Date
Amber Brown
b2b90b7d34
Hawkowl/fix missing auth (#5328) 2019-06-04 15:54:27 +10:00
Richard van der Hoff
c5d60eadd5 Notary server: make requests to origins in parallel 
... else we're guaranteed to time out.
 2019-06-04 00:16:56 +01:00
Amber Brown
2889b05554
Unify v1 and v2 REST client APIs (#5226) 2019-06-03 21:28:59 +10:00
Alexander Trost
dc3e586938
SAML2 Improvements and redirect stuff 
Signed-off-by: Alexander Trost <galexrt@googlemail.com>
 2019-06-02 18:14:40 +02:00
Eisha Chen-yen-su
8824325b82 Fix ignored filter field in /messages endpoint 
This fixes a bug which were causing the "event_format" field to be
ignored in the filter of requests to the `/messages` endpoint of the
CS API.

Signed-off-by: Eisha Chen-yen-su <chenyensu0@gmail.com>
 2019-05-30 16:58:53 +02:00
Erik Johnston
45f5d8f3fd
Merge pull request #5256 from aaronraimist/logout-correct-error 
Show correct error when logging out and access token is missing
 2019-05-30 13:33:44 +01:00
Aaron Raimist
123918b739
Lint 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-05-29 14:44:28 -05:00
Amber Brown
46c8f7a517
Implement the SHHS complexity API (#5216) 2019-05-30 01:47:16 +10:00
Amber Brown
532b825ed9
Serve CAS login over r0 (#5286) 2019-05-30 00:55:18 +10:00
Aaron Raimist
30858ff461 Fix error when downloading thumbnail with width/height param missing (#5258) 
Fix error when downloading thumbnail with width/height param missing

Fixes #2748

Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-05-29 14:27:41 +01:00
Aaron Raimist
119c9c10b0
Get rid of try except 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-05-27 00:13:56 -05:00
Aaron Raimist
56f07d980a
Show correct error when logging out and access token is missing 
Signed-off-by: Aaron Raimist <aaron@raim.ist>
 2019-05-24 16:32:27 -05:00
Erik Johnston
d16f5574b6
Merge pull request #5220 from matrix-org/erikj/dont_bundle_live_events 
Don't bundle aggregations with events in /sync or /events or state queries
 2019-05-24 10:36:31 +01:00
Erik Johnston
8c41c04ee4
Merge pull request #5244 from matrix-org/rav/server_keys/00-factor-out-fetchers 
Factor out KeyFetchers from KeyRing
 2019-05-23 16:28:51 +01:00
Andrew Morgan
6368150a74
Add config option for setting homeserver's default room version (#5223) 
Replaces DEFAULT_ROOM_VERSION constant with a method that first checks the config, then returns a hardcoded value if the option is not present.

That hardcoded value is now located in the server.py config file.
 2019-05-23 15:00:20 +01:00
Richard van der Hoff
ec24108cc2 Fix remote_key_resource 2019-05-23 14:52:13 +01:00
Erik Johnston
de7672b78f Don't bundle events in /sync or /events 
As we'll send down the annotations too anyway, so this just ends up
confusing clients.
 2019-05-21 13:54:09 +01:00
Erik Johnston
57ba3451b6
Merge pull request #5209 from matrix-org/erikj/reactions_base 
Land basic reaction and edit support.
 2019-05-20 14:06:40 +01:00
PauRE
f89f688a55 Fix image orientation when generating thumbnail (#5039) 2019-05-16 19:04:26 +01:00
David Baker
07cff7b121
Merge pull request #5174 from matrix-org/dbkr/add_dummy_flow_to_recaptcha_only 
Re-order registration stages to do msisdn & email auth last
 2019-05-16 17:27:39 +01:00
Erik Johnston
7a7eba8302 Move parsing of tokens out of storage layer 2019-05-16 14:26:23 +01:00
Erik Johnston
95f3fcda3c Check that event is visible in new APIs 2019-05-16 14:19:06 +01:00
David Baker
fafb936de5
Merge pull request #5187 from matrix-org/dbkr/only_check_threepid_not_in_use_if_actually_registering 
Only check 3pids not in use when registering
 2019-05-16 10:58:09 +01:00
Erik Johnston
a0603523d2 Add aggregations API 2019-05-16 09:37:20 +01:00
Amber Brown
f1e5b41388
Make all the rate limiting options more consistent (#5181) 2019-05-15 12:06:04 -05:00
Richard van der Hoff
5f027a315f
Drop support for v2_alpha API prefix (#5190) 2019-05-15 17:37:46 +01:00
Erik Johnston
e6459c26b4 Actually implement idempotency 2019-05-15 17:28:33 +01:00
Erik Johnston
b50641e357 Add simple pagination API 2019-05-15 13:36:51 +01:00
Erik Johnston
efe3c7977a Add simple send_relation API and track in DB 2019-05-15 13:36:51 +01:00
David Baker
6ca88c4693 Only check 3pids not in use when registering 
We checked that 3pids were not already in use before we checked if
we were going to return the account previously registered in the
same UI auth session, in which case the 3pids will definitely
be in use.

https://github.com/vector-im/riot-web/issues/9586
 2019-05-14 19:04:59 +01:00
Erik Johnston
a80e6b53f9 Newsfile 2019-05-14 13:12:23 +01:00
Erik Johnston
b54b03f9e1 Allow client event serialization to be async 2019-05-14 11:58:01 +01:00
Amber Brown
df2ebd75d3
Migrate all tests to use the dict-based config format instead of hanging items off HomeserverConfig (#5171) 2019-05-13 15:01:14 -05:00
David Baker
9e99143c47 Merge remote-tracking branch 'origin/develop' into dbkr/add_dummy_flow_to_recaptcha_only 2019-05-13 15:37:03 +01:00
Andrew Morgan
2f48c4e1ae
URL preview blacklisting fixes (#5155) 
Prevents a SynapseError being raised inside of a IResolutionReceiver and instead opts to just return 0 results. This thus means that we have to lump a failed lookup and a blacklisted lookup together with the same error message, but the substitute should be generic enough to cover both cases.
 2019-05-10 10:32:44 -07:00
David Baker
04299132af Re-order flows so that email auth is done last 
It's more natural for the user if the bit that takes them away
from the registration flow comes last. Adding the dummy stage allows
us to do the stages in this order without the ambiguity.
 2019-05-10 13:58:03 +01:00
David Baker
9c61dce3c8 Comment 2019-05-10 11:14:55 +01:00
David Baker
8714ff6d51 Add a DUMMY stage to captcha-only registration flow 
This allows the client to complete the email last which is more
natual for the user. Without this stage, if the client would
complete the recaptcha (and terms, if enabled) stages and then the
registration request would complete because you've now completed a
flow, even if you were intending to complete the flow that's the
same except has email auth at the end.

Adding a dummy auth stage to the recaptcha-only flow means it's
always unambiguous which flow the client was trying to complete.
Longer term we should think about changing the protocol so the
client explicitly says which flow it's trying to complete.

vector-im/riot-web#9586
 2019-05-10 11:09:53 +01:00
David Baker
c2bb7476c9 Revert 085ae346ac 
Accidentally went straight to develop
 2019-05-10 11:08:01 +01:00
David Baker
085ae346ac Add a DUMMY stage to captcha-only registration flow 
This allows the client to complete the email last which is more
natual for the user. Without this stage, if the client would
complete the recaptcha (and terms, if enabled) stages and then the
registration request would complete because you've now completed a
flow, even if you were intending to complete the flow that's the
same except has email auth at the end.

Adding a dummy auth stage to the recaptcha-only flow means it's
always unambiguous which flow the client was trying to complete.
Longer term we should think about changing the protocol so the
client explicitly says which flow it's trying to complete.

https://github.com/vector-im/riot-web/issues/9586
 2019-05-10 10:52:24 +01:00
Matthew Hodgson
c0e0740bef add options to require an access_token to GET /profile and /publicRooms on CS API (#5083) 
This commit adds two config options:

* `restrict_public_rooms_to_local_users`

Requires auth to fetch the public rooms directory through the CS API and disables fetching it through the federation API.

* `require_auth_for_profile_requests`

When set to `true`, requires that requests to `/profile` over the CS API are authenticated, and only returns the user's profile if the requester shares a room with the profile's owner, as per MSC1301.

MSC1301 also specifies a behaviour for federation (only returning the profile if the server asking for it shares a room with the profile's owner), but that's currently really non-trivial to do in a not too expensive way. Next step is writing down a MSC that allows a HS to specify which user sent the profile query. In this implementation, Synapse won't send a profile query over federation if it doesn't believe it already shares a room with the profile's owner, though.

Groups have been intentionally omitted from this commit.
 2019-05-08 18:26:56 +01:00
Richard van der Hoff
59e2d2694d
Remove the requirement to authenticate for /admin/server_version. (#5122) 
This endpoint isn't much use for its intended purpose if you first need to get
yourself an admin's auth token.

I've restricted it to the `/_synapse/admin` path to make it a bit easier to
lock down for those concerned about exposing this information. I don't imagine
anyone is using it in anger currently.
 2019-05-07 09:29:30 +01:00
Richard van der Hoff
12f9d51e82
Add admin api for sending server_notices (#5121) 2019-05-02 11:59:16 +01:00
Richard van der Hoff
0836cbb9f5
Factor out an "assert_requester_is_admin" function (#5120) 
Rather than copying-and-pasting the same four lines hundreds of times
 2019-05-02 10:45:52 +01:00
Richard van der Hoff
f203c98794 fix examples 2019-05-01 17:49:56 +01:00
Richard van der Hoff
40e576e29c Move admin api impl to its own package 
It doesn't really belong under rest/client/v1 any more.
 2019-05-01 15:44:30 +01:00
Richard van der Hoff
8e9ca83537 Move admin API to a new prefix 2019-05-01 15:44:30 +01:00
Richard van der Hoff
579b637b6c Move admin API away from ClientV1RestServlet 2019-05-01 15:16:04 +01:00
Michael Kaye
788163e204 Remove log error for .well-known/matrix/client (#4972) 2019-04-24 17:44:06 +01:00
Amber Brown
6b2b9a58c4 Prevent "producer not unregistered" message (#5009) 2019-04-24 17:37:32 +01:00
Erik Johnston
ca90336a69 Merge branch 'develop' of github.com:matrix-org/synapse into babolivier/account_expiration 2019-04-17 19:44:40 +01:00
Brendan Abolivier
eaf41a943b Add management endpoints for account validity 2019-04-17 19:34:45 +01:00
Brendan Abolivier
91934025b9
Merge pull request #5047 from matrix-org/babolivier/account_expiration 
Send out emails with links to extend an account's validity period
 2019-04-17 14:57:39 +01:00
Brendan Abolivier
20f0617e87 Send out emails with links to extend an account's validity period 2019-04-17 14:42:20 +01:00
Brendan Abolivier
49ff74da9b
Merge pull request #5071 from matrix-org/babolivier/3pid-check 
Make sure we're not registering the same 3pid twice
 2019-04-17 14:37:42 +01:00
Brendan Abolivier
600ec04739 Make sure we're not registering the same 3pid twice 2019-04-17 14:23:01 +01:00
Erik Johnston
fd2fcb817c
Merge pull request #5070 from matrix-org/erikj/postpath 
Remove usage of request.postpath
 2019-04-17 09:03:26 +01:00
Erik Johnston
ad010f6306 Remove usage of request.postpath 
This is an undocumented variable in twisted, and relies on the servlet
being mounted in the right way.

This also breaks getting push rules on workers.
 2019-04-16 17:42:50 +01:00
Erik Johnston
6e27a8620f
Merge pull request #5063 from matrix-org/erikj/move_endpoints 
Move some rest endpoints to client reader
 2019-04-15 18:55:01 +01:00
Erik Johnston
ec638a1602 Only handle GET requests for /push_rules 2019-04-15 18:51:48 +01:00
Andrew Morgan
caa76e6021
Remove periods from copyright headers (#5046) 2019-04-11 17:08:13 +01:00
Erik Johnston
c192bf8970 Add admin API for group deletion 2019-04-03 16:29:52 +01:00
Andrew Morgan
4a4d5c4fd6
Fix grammar and document get_current_users_in_room (#4998) 2019-04-03 14:32:20 +01:00
Neil Johnson
e8419554ff
Remove presence lists (#4989) 
Remove presence list support as per MSC 1819
 2019-04-03 11:11:15 +01:00
Erik Johnston
8f549c1177
Merge pull request #4982 from matrix-org/erikj/msc1915 
Implement MSC1915 - 3PID unbind APIs
 2019-04-03 11:07:09 +01:00
Erik Johnston
862d6e5ba5 Add unbind API to /r0 as it is now stabalised 2019-04-01 15:25:19 +01:00
Erik Johnston
057715aaa2 Allowing specifying IS to use in unbind API. 
By default the homeserver will use the identity server used during the
binding of the 3PID to unbind the 3PID. However, we need to allow
clients to explicitly ask the homeserver to unbind via a particular
identity server, for the case where the 3PID was bound out of band from
the homeserver.

Implements MSC915.
 2019-04-01 15:25:18 +01:00
Richard van der Hoff
54a87a7b08
Collect room-version variations into one place (#4969) 
Collect all the things that make room-versions different to one another into
one place, so that it's easier to define new room versions.
 2019-04-01 10:24:38 +01:00
Neil Johnson
c7296bcb98
remove log line for password (#4965) 
Remove log line for password.
 2019-03-28 17:38:01 +00:00
Andrew Morgan
bbd244c7b2
Support 3PID login in password providers (#4931) 
Adds a new method, check_3pid_auth, which gives password providers
the chance to allow authentication with third-party identifiers such
as email or msisdn.
 2019-03-26 17:48:30 +00:00
Erik Johnston
3959858eaa
Merge pull request #4904 from matrix-org/erikj/fix_shutdown 
Fixup shutdown room API
 2019-03-21 11:24:42 +00:00
Erik Johnston
aa959a6c07 Use flags 2019-03-20 17:40:29 +00:00
Erik Johnston
72a14860ab Gracefully handle failing to kick user 2019-03-20 16:54:21 +00:00
Erik Johnston
6b28890543 Log new room ID 2019-03-20 16:52:28 +00:00
Erik Johnston
320667a479 Add option to disable searching in the user dir 
We still populate it, as it can still be accessed via the admin API.
 2019-03-19 16:40:19 +00:00
Brendan Abolivier
899e523d6d
Add ratelimiting on login (#4821) 
Add two ratelimiters on login (per-IP address and per-userID).
 2019-03-15 17:46:16 +00:00
Matthew Hodgson
2326e00bc4 fix incorrect encoding of filenames with spaces in (#2090) 
fixes https://github.com/vector-im/riot-web/issues/3155
 2019-03-11 09:53:45 +00:00
Erik Johnston
b879870b2d Send message after room has been shutdown 
Currently the explanation message is sent to the abuse room before any
users are forced joined, which means it tends to get lost in the backlog
of joins.

So instead we send the message *after* we've forced joined everyone.
 2019-03-06 17:35:11 +00:00
Brendan Abolivier
6f3cde8b25 Make registration ratelimiter separate from the main events one 2019-03-06 11:02:42 +00:00
Brendan Abolivier
f4195f4118 Revert "Split ratelimiters in two (one for events, one for registration)" 
This reverts commit d7dbad3526.
 2019-03-06 10:55:22 +00:00
Brendan Abolivier
d7dbad3526 Split ratelimiters in two (one for events, one for registration) 2019-03-05 18:41:27 +00:00
Erik Johnston
16c8b4ecbd
Merge pull request #4772 from jbweston/jbweston/server-version-api 
Add 'server_version' endpoint to admin API
 2019-03-05 16:31:00 +00:00
Brendan Abolivier
a4c3a361b7
Add rate-limiting on registration (#4735) 
* Rate-limiting for registration

* Add unit test for registration rate limiting

* Add config parameters for rate limiting on auth endpoints

* Doc

* Fix doc of rate limiting function

Co-Authored-By: babolivier <contact@brendanabolivier.com>

* Incorporate review

* Fix config parsing

* Fix linting errors

* Set default config for auth rate limiting

* Fix tests

* Add changelog

* Advance reactor instead of mocked clock

* Move parameters to registration specific config and give them more sensible default values

* Remove unused config options

* Don't mock the rate limiter un MAU tests

* Rename _register_with_store into register_with_store

* Make CI happy

* Remove unused import

* Update sample config

* Fix ratelimiting test for py2

* Add non-guest test
 2019-03-05 14:25:33 +00:00
Joseph Weston
1e8388b311 Add 'server_version' endpoint to admin API 
This is required because the 'Server' HTTP header is not always
passed through proxies.
 2019-03-01 09:56:58 +01:00
Richard van der Hoff
68f47d6744 Fix parsing of Content-Disposition headers (#4763) 
* Fix parsing of Content-Disposition headers

TIL: filenames in content-dispostion headers can contain semicolons, and aren't
%-encoded.

* fix python2 incompatibility

* Fix docstrings
 2019-02-27 14:29:10 -08:00
Erik Johnston
899a119c2b Don't log stack trace when client has gone away during media download (#4738) 
* Don't log stack trace when client has gone away during media download

* Newsfile

* Fixup newsfile
 2019-02-25 11:17:22 -08:00
Matthew Hodgson
70ea2f4e1d switch from google.com to recaptcha.net for reCAPTCHA (#4731) 
* add trivial clarification about jemalloc

* switch from google.com to recaptcha.net

because https://developers.google.com/recaptcha/docs/faq#can-i-use-recaptcha-globally
 2019-02-25 11:15:36 -08:00
Erik Johnston
dbdc565dfd Fix registration on workers (#4682) 
* Move RegistrationHandler init to HomeServer

* Move post registration actions to RegistrationHandler

* Add post regisration replication endpoint

* Newsfile
 2019-02-20 18:47:31 +11:00
Erik Johnston
fc2c245a1f
Merge pull request #4666 from matrix-org/erikj/register_login_split 
Split out registration to worker
 2019-02-18 17:18:06 +00:00
Erik Johnston
af691e415c Move register_device into handler 2019-02-18 16:49:38 +00:00
Erik Johnston
092b541401 Fixup 2019-02-18 15:52:26 +00:00
Erik Johnston
8b9ae6d3a6 Update docs 2019-02-18 15:26:13 +00:00
Matthew Hodgson
04dad5ac16 fix missig import 2019-02-18 15:59:23 +01:00
Erik Johnston
eb2b8523ae Split out registration to worker 
This allows registration to be handled by a worker, though the actual
write to the database still happens on master.

Note: due to the in-memory session map all registration requests must be
handled by the same worker.
 2019-02-18 12:12:57 +00:00
Matthew Hodgson
bfcefbb230 tabs 2019-02-14 19:53:12 +00:00
Matthew Hodgson
6f47bc3fb2 set CORS on .well-known URI to unbreak modular 
otherwise a riot/web running on foo.riot.im can't query
the .well-known on foo.modular.im...
 2019-02-14 19:51:41 +00:00
Erik Johnston
719e073f00
Merge pull request #4580 from matrix-org/uhoreg/e2e_backup_add_updating 
add updating of backup versions
 2019-02-11 13:45:49 +00:00
Valentin Anger
2dc2b6e9f1 Allow "unavailable" presence status for /sync (#4592) 
* Allow "unavailable" presence status for /sync

Closes #3772, closes #3779

Signed-off-by: Valentin Anger <valentin.an.1999@gmail.com>

* Add changelog for PR 4592
 2019-02-08 21:09:56 +00:00
Hubert Chathi
afae8442b5 make sure version is in body and wrap in linearizer queue 
also add tests
 2019-02-08 01:32:45 -05:00
Hubert Chathi
8248637173 add new endpoint to update backup versions 2019-02-06 17:57:10 -05:00
Neil Johnson
b37e8c9572 Merge branch 'neilj/room_capabilities' of github.com:matrix-org/synapse into neilj/room_capabilities 2019-01-30 10:56:47 +00:00
Neil Johnson
f834d98402 isort 2019-01-30 10:55:42 +00:00
Neil Johnson
ee4df7fd7a
Merge branch 'develop' into neilj/room_capabilities 2019-01-30 10:28:08 +00:00
Neil Johnson
9c850d9d5e formatting and use constants where available 2019-01-30 10:23:26 +00:00
Neil Johnson
2f46804055 Populate default room version from Constants 2019-01-30 09:39:10 +00:00
Neil Johnson
c7837dce24 reflect that rooms v3 is a stable room version 2019-01-30 09:33:30 +00:00
Neil Johnson
e4bef9d470 rework format of change password capability 2019-01-29 18:04:56 +00:00
Neil Johnson
f03b3a7a3a support change_password in capabilities end-point 2019-01-29 15:58:37 +00:00
Andrew Morgan
e65a17b26f Check consent dir path on startup 2019-01-29 15:30:33 +00:00
Neil Johnson
a124025dab enforce auth for capabilities endpoint 2019-01-29 11:37:56 +00:00
Neil Johnson
327b992e17 register capabilities servlet 2019-01-29 10:28:35 +00:00
Neil Johnson
893107be78 backout v3 2019-01-25 17:27:36 +00:00
Neil Johnson
95f871fc0d Support room version capabilities in CS API (MSC1804) 2019-01-25 11:16:29 +00:00
Andrew Morgan
821b65aeb5 Merge branch 'develop' of github.com:matrix-org/synapse into anoa/dm_room_upgrade 2019-01-25 11:09:53 +00:00
Neil Johnson
c99c2d58d7 move guard out of is_threepid_reserved and into register.py 2019-01-22 17:47:00 +00:00
Neil Johnson
d619b113ed Fix None guard in config.server.is_threepid_reserved 2019-01-22 16:52:29 +00:00
Andrew Morgan
25d64a846a Fix typos 2019-01-22 11:00:04 +00:00
Neil Johnson
5349262302 Config option to disable requesting MSISDN on registration 2019-01-21 14:59:37 +00:00
Erik Johnston
b970cb0e96 Refactor request sending to have better excpetions (#4358) 
* Correctly retry and back off if we get a HTTPerror response

* Refactor request sending to have better excpetions

MatrixFederationHttpClient blindly reraised exceptions to the caller
without differentiating "expected" failures (e.g. connection timeouts
etc) versus more severe problems (e.g. programming errors).

This commit adds a RequestSendFailed exception that is raised when
"expected" failures happen, allowing the TransactionQueue to log them as
warnings while allowing us to log other exceptions as actual exceptions.
 2019-01-08 11:04:28 +00:00
Travis Ralston
bc1fa8cd01 Add GET account data routes (#4303) 
As per https://github.com/matrix-org/matrix-doc/issues/1339
 2019-01-07 10:27:54 +00:00
Amber Brown
d7843f47b6
Remove v1 only REST APIs now we don't ship matrix console (#4334) 2018-12-29 23:12:30 +11:00
Richard van der Hoff
f85676cc93
Return well_known in /login response (#4319) 
... as per MSC1730.
 2018-12-24 10:44:33 +01:00
Amber Brown
ea6abf6724
Fix IP URL previews on Python 3 (#4215) 2018-12-22 01:56:13 +11:00
Neil Johnson
d2f7c4e6b1
create support user (#4141) 
Allow for the creation of a support user.

A support user can access the server, join rooms, interact with other users, but does not appear in the user directory nor does it contribute to monthly active user limits.
 2018-12-14 18:20:59 +00:00
David Baker
89ac2a5bdb Add 'sandbox' to CSP for media repo (#4284) 
* Add 'sandbox' to the CSP for media repo

* Changelog
 2018-12-11 04:05:02 +11:00
Will Hunt
fee831c040 Move imports to one line 2018-12-10 13:52:33 +00:00
Will Hunt
466c1f3e01
Use send_cors 2018-12-10 13:11:37 +00:00
Will Hunt
91206e09f2 changelog & isort 2018-12-09 17:39:44 +00:00
Will Hunt
dbf736ba66
Make /config more CORS-y 2018-12-09 13:27:22 +00:00
Richard van der Hoff
30da50a5b8
Initialise user displayname from SAML2 data (#4272) 
When we register a new user from SAML2 data, initialise their displayname
correctly.
 2018-12-07 14:44:46 +01:00
Richard van der Hoff
c7401a697f
Implement SAML2 authentication (#4267) 
This implements both a SAML2 metadata endpoint (at
`/_matrix/saml2/metadata.xml`), and a SAML2 response receiver (at
`/_matrix/saml2/authn_response`). If the SAML2 response matches what's been
configured, we complete the SSO login flow by redirecting to the client url
(aka `RelayState` in SAML2 jargon) with a login token.

What we don't yet have is anything to build a SAML2 request and redirect the
user to the identity provider. That is left as an exercise for the reader.
 2018-12-07 13:11:11 +01:00
Richard van der Hoff
c588b9b9e4
Factor SSO success handling out of CAS login (#4264) 
This is mostly factoring out the post-CAS-login code to somewhere we can reuse
it for other SSO flows, but it also fixes the userid mapping while we're at it.
 2018-12-07 13:10:07 +01:00
Richard van der Hoff
b0c24a66ec Rip out half-implemented m.login.saml2 support (#4265) 
* Rip out half-implemented m.login.saml2 support

This was implemented in an odd way that left most of the work to the client, in
a way that I really didn't understand. It's going to be a pain to maintain, so
let's start by ripping it out.

* drop undocumented dependency on dateutil

It turns out we were relying on dateutil being pulled in transitively by
pysaml2. There's no need for that bloat.
 2018-12-06 19:44:38 +11:00
Richard van der Hoff
e8d98466b0
Implement .well-known handling (#4262) 
Sometimes it's useful for synapse to generate its own .well-known file.
 2018-12-05 14:38:58 +01:00
Amber Brown
fd96dd75a3 Fix non-ASCII pushrules (#4248) 2018-12-04 12:44:02 +01:00
Amber Brown
d3c61ef906 fix type error 2018-12-03 22:27:41 +11:00
Neil Johnson
7039ece8fb Neilj/fix autojoin (#4223) 
* Fix auto join failures for servers that require user consent

* Fix auto join failures for servers that require user consent
 2018-11-28 22:24:57 +11:00
Richard van der Hoff
944d524f18 Support m.login.sso (#4220) 
* Clean up the CSS for the fallback login form

I was finding this hard to work with, so simplify a bunch of things. Each
flow is now a form inside a div of class login_flow.

The login_flow class now has a fixed width, as that looks much better than each
flow having a differnt width.

* Support m.login.sso

MSC1721 renames m.login.cas to m.login.sso. This implements the change
(retaining support for m.login.cas for older clients).

* changelog
 2018-11-27 18:51:52 +11:00
Amber Brown
80cac86b2c
Fix fallback auth on Python 3 (#4197) 2018-11-19 12:27:33 -06:00
Travis Ralston
ab4526a153 Remove duplicate slashes in generated consent URLs 2018-11-15 20:41:53 -07:00
Amber Brown
8b1affe7d5
Fix Content-Disposition in media repository (#4176) 2018-11-15 15:55:58 -06:00
Amber Brown
df758e155d
Use <meta> tags to discover the per-page encoding of html previews (#4183) 2018-11-15 11:05:08 -06:00
Richard van der Hoff
c1efcd7c6a Add a test for the public T&Cs form 2018-11-14 10:46:27 +00:00
Richard van der Hoff
83a5f459aa Fix an internal server error when viewing the public privacy policy 2018-11-14 10:21:07 +00:00
hera
2b075fb03a Fix encoding error for consent form on python3 
The form was rendering this as "b'01234....'".

-- richvdh
 2018-11-08 11:05:39 +00:00
Amber Brown
b3708830b8
Fix URL preview bugs (type error when loading cache from db, content-type including quotes) (#4157) 2018-11-08 01:37:43 +11:00
Travis Ralston
0f5e51f726 Add config variables for enabling terms auth and the policy name (#4142) 
So people can still collect consent the old way if they want to.
 2018-11-06 10:32:34 +00:00
Hubert Chathi
f1087106cf
handle empty backups according to latest spec proposal (#4123) 
fixes #4056
 2018-11-05 17:59:29 -05:00
Amber Brown
efdcbbe46b
Tests for user consent resource (#4140) 2018-11-06 05:53:44 +11:00
Travis Ralston
642505abc3 Fix logic error that prevented guests from seeing the privacy policy 2018-11-01 16:48:32 -06:00
Travis Ralston
a8d41c6aff Include a version query string arg for the consent route 2018-10-31 13:19:28 -06:00
Travis Ralston
d1e7b9c44c Merge branch 'develop' into travis/login-terms 2018-10-31 13:15:14 -06:00
Amber Brown
f79f454485
Remove deprecated v1 key exchange endpoint (#4119) 2018-10-31 22:29:02 +11:00
Amber Brown
4cd1c9f2ff
Delete the disused & unspecced identicon functionality (#4106) 2018-10-29 23:57:24 +11:00
Richard van der Hoff
193cadc988 Address review comments 
Improve comments, get old room state from the context we already have
 2018-10-26 17:10:30 +01:00
Richard van der Hoff
0f7d1c9906 Basic initial support for room upgrades 
Currently just creates a new, empty, room, and sends a tombstone in the old
room.
 2018-10-25 19:10:24 +01:00
Erik Johnston
cb53ce9d64
Refactor state group lookup to reduce DB hits (#4011) 
Currently when fetching state groups from the data store we make two
hits two the database: once for members and once for non-members (unless
request is filtered to one or the other). This adds needless load to the
datbase, so this PR refactors the lookup to make only a single database
hit.
 2018-10-25 17:49:55 +01:00
Travis Ralston
54def42c19 Merge branch 'develop' into travis/login-terms 2018-10-24 13:22:59 -06:00
Richard van der Hoff
ef771cc4c2 Fix a number of flake8 errors 
Broadly three things here:

* disable W504 which seems a bit whacko
* remove a bunch of `as e` expressions from exception handlers that don't use
  them
* use `r""` for strings which include backslashes

Also, we don't use pep8 any more, so we can get rid of the duplicate config
there.
 2018-10-24 10:39:03 +01:00
Erik Johnston
cb23aa4c42
Merge pull request #4063 from matrix-org/erikj/cleaup_alias_creation 
Clean up room alias creation
 2018-10-24 09:40:27 +01:00
Richard van der Hoff
5c445114d3
Correctly account for cpu usage by background threads (#4074) 
Wrap calls to deferToThread() in a thing which uses a child logcontext to
attribute CPU usage to the right request.

While we're in the area, remove the logcontext_tracer stuff, which is never
used, and afaik doesn't work.

Fixes #4064
 2018-10-23 13:12:32 +01:00
Erik Johnston
74e7617083 Clean up room alias creation 2018-10-19 10:11:56 +01:00
Travis Ralston
49a044aa5f Merge branch 'develop' into travis/login-terms 2018-10-18 09:57:58 -06:00
Erik Johnston
f6a0a02a62 Fix bug where we raised StopIteration in a generator 
This made python 3.7 unhappy
 2018-10-17 16:10:52 +01:00
Travis Ralston
a8ed93a4b5 pep8 2018-10-15 16:10:29 -06:00
Travis Ralston
f293d124b6 Merge branch 'develop' into travis/login-terms 2018-10-15 14:44:32 -06:00
Travis Ralston
5119818e9d Rely on the lack of ?u to represent public access 
also general cleanup
 2018-10-12 18:03:17 -06:00
David Baker
a45f2c3a00 missed one 2018-10-12 14:33:55 +01:00
David Baker
bddfad253a Don't mangle exceptions 2018-10-12 11:48:02 +01:00
David Baker
86ef9760a7 Split /room_keys/version into 2 servlets 2018-10-12 11:35:08 +01:00
David Baker
83e72bb2f0 PR feedback pt. 1 2018-10-12 11:26:18 +01:00
David Baker
d3464ce708 isort 2018-10-09 10:33:59 +01:00
David Baker
dc045ef202 Merge remote-tracking branch 'origin/develop' into dbkr/e2e_backups 2018-10-09 10:05:02 +01:00
Travis Ralston
537d0b7b36 Use a flag rather than a new route for the public policy 
This also means that the template now has optional parameters, which will need to be documented somehow.
 2018-10-03 17:50:11 -06:00
Travis Ralston
f9d34a763c Auto-consent to the privacy policy if the user registered with terms 2018-10-03 17:39:45 -06:00
Travis Ralston
dfcad5fad5 Make the terms flow requried 2018-10-03 17:39:00 -06:00
Travis Ralston
3099d96dba Flesh out the fallback auth for terms 2018-10-03 17:39:00 -06:00
Travis Ralston
fd99787162 Incorporate Dave's work for GDPR login flows 
As per https://github.com/vector-im/riot-web/issues/7168#issuecomment-419996117
 2018-10-03 15:57:42 -06:00
Richard van der Hoff
4c3e7eeec5
Merge pull request #3932 from matrix-org/erikj/auto_start_expiring_caches 
Fix some instances of ExpiringCache not expiring cache items
 2018-09-25 12:02:57 +01:00
Jérémy Farnaud
6cf261930a added "media-src: 'self'" to CSP for resources (#3578) 
Synapse doesn’t allow for media resources to be played directly from
Chrome. It is a problem for users on other networks (e.g. IRC)
communicating with Matrix users through a gateway. The gateway sends
them the raw URL for the resource when a Matrix user uploads a video
and the video cannot be played directly in Chrome using that URL.

Chrome argues it is not authorized to play the video because of the
Content Security Policy. Chrome checks for the "media-src" policy which
is missing, and defauts to the "default-src" policy which is "none".

As Synapse already sends "object-src: 'self'" I thought it wouldn’t be
a problem to add "media-src: 'self'" to the CSP to fix this problem.
 2018-09-25 11:55:02 +01:00
Erik Johnston
8601c24287 Fix some instances of ExpiringCache not expiring cache items 
ExpiringCache required that `start()` be called before it would actually
start expiring entries. A number of places didn't do that.

This PR removes `start` from ExpiringCache, and automatically starts
backround reaping process on creation instead.
 2018-09-21 14:19:46 +01:00
David Baker
bc74925c5b WIP e2e key backups 
Continues from uhoreg's branch

This just fixed the errcode on /room_keys/version if no backup and
updates the schema delta to be on the latest so it gets run
 2018-09-13 17:02:59 +01:00
Amber Brown
546aee7e52
Merge pull request #3835 from krombel/fix_3821 
fix VOIP crashes under Python 3
 2018-09-12 20:44:18 +10:00
Amber Brown
02aa41809b
Port rest/ to Python 3 (#3823) 2018-09-12 20:41:31 +10:00
Krombel
7bc22539ff fix VOIP crashes under Python 3 (#3821) 2018-09-10 14:30:08 +02:00
Hubert Chathi
3801b8aa03 try to make flake8 and isort happy 2018-09-06 11:35:19 -04:00
Richard van der Hoff
dfba1d843d
Merge pull request #3790 from matrix-org/rav/respect_event_format_in_filter 
Implement 'event_format' filter param in /sync
 2018-09-05 16:24:14 +01:00
Richard van der Hoff
87c18d12ee Implement 'event_format' filter param in /sync 
This has been specced and part-implemented; let's implement it for /sync (but
no other endpoints yet :/).
 2018-09-04 15:20:09 +01:00
Neil Johnson
bae37cd811 improve human readable error message 2018-09-04 12:07:00 +01:00
Neil Johnson
c42f7fd7b9 improve human readable error messages 2018-09-04 12:03:17 +01:00
Neil Johnson
0b01281e77 move threepid checker to config, add missing yields 2018-08-31 17:11:11 +01:00
Neil Johnson
09f3cf1a7e ensure post registration auth checks do not fail erroneously 2018-08-31 15:42:51 +01:00
Neil Johnson
ea068d6f3c fix bug where preserved threepid user comes to sign up and server is mau blocked 2018-08-31 10:49:14 +01:00
Hubert Chathi
83caead95a
Merge branch 'develop' into e2e_backups 2018-08-24 11:44:26 -04:00
Erik Johnston
db10f553ba
Merge pull request #3724 from Half-Shot/hs/guest-fetch-event 
Allow guests to use /rooms/:roomId/event/:eventId
 2018-08-22 13:41:08 +01:00
Richard van der Hoff
f7bf181a90 fix another consent encoding fail 2018-08-21 23:14:25 +01:00
Richard van der Hoff
f7baff6f7b Fix 500 error from /consent form 
Fixes #3731
 2018-08-21 22:47:07 +01:00
Will Hunt
f89f6b7c09 Allow guests to access /rooms/:roomId/event/:eventId 2018-08-20 18:25:54 +01:00
Amber Brown
324525f40c
Port over enough to get some sytests running on Python 3 (#3668) 2018-08-20 23:54:49 +10:00
Amber Brown
c334ca67bb
Integrate presence from hotfixes (#3694) 2018-08-18 01:08:45 +10:00
Amber Brown
04f5d2db62
Remove v1/register's broken shared secret functionality (#3703) 2018-08-18 00:55:01 +10:00
Will Hunt
c151b32b1d Add GET media/v1/config (#3184) 2018-08-16 14:23:38 +01:00
Matthew Hodgson
762a758fea lazyload aware /messages (#3589) 2018-08-16 14:22:47 +01:00
Matthew Hodgson
3f543dc021 initial cut at a room summary API (#3574) 2018-08-16 09:46:50 +01:00
Matthew Hodgson
2f78f432c4 speed up /members and add at= and membership params (#3568) 2018-08-15 16:35:22 +01:00
Erik Johnston
fef2e65d12
Merge pull request #3667 from matrix-org/erikj/fixup_unbind 
Don't fail requests to unbind 3pids for non supporting ID servers
 2018-08-15 10:32:12 +01:00
Matthew Hodgson
4f7064f6b5 missing import 2018-08-12 19:14:31 -04:00
Matthew Hodgson
54ac18e832 use parse_string 2018-08-12 19:14:31 -04:00
Matthew Hodgson
72788cf9c1 support DELETE /version with no args 2018-08-12 19:14:31 -04:00
Matthew Hodgson
fe87890b18 implement remaining tests and make them work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
93d174bcc4 improve docstring 2018-08-12 19:14:31 -04:00
Matthew Hodgson
14b3da63a3 add a tonne of docstring; make upload_room_keys properly assert version 2018-08-12 19:14:31 -04:00
Matthew Hodgson
9f500cb39e more docstring for the e2e_room_keys rest 2018-08-12 19:14:31 -04:00
Matthew Hodgson
0abb205b47 blindly incorporate PR review - needs testing & fixing 2018-08-12 19:14:31 -04:00
Matthew Hodgson
69e51c7ba4 make /room_keys/version work 2018-08-12 19:14:31 -04:00
Matthew Hodgson
8ae64b270f implement /room_keys/version too (untested) 2018-08-12 19:14:31 -04:00
Matthew Hodgson
cf1e2000f6 document the API 2018-08-12 19:13:09 -04:00
Matthew Hodgson
6b8c07abc2 make it work and fix pep8 2018-08-12 19:13:09 -04:00
Matthew Hodgson
0bc4627a73 interim WIP checkin; doesn't build yet 2018-08-12 18:23:10 -04:00
Matthew Hodgson
53ace904b2 total WIP skeleton for /room_keys API 2018-08-12 18:23:10 -04:00
Amber Brown
b37c472419
Rename async to async_helpers because async is a keyword on Python 3.7 (#3678) 2018-08-10 23:50:21 +10:00
Erik Johnston
360ba89c50 Don't fail requests to unbind 3pids for non supporting ID servers 
Older identity servers may not support the unbind 3pid request, so we
shouldn't fail the requests if we received one of 400/404/501. The
request still fails if we receive e.g. 500 responses, allowing clients
to retry requests on transient identity server errors that otherwise do
support the API.

Fixes #3661
 2018-08-08 12:06:18 +01:00
Richard van der Hoff
1fa98495d0
Merge pull request #3639 from matrix-org/rav/refactor_error_handling 
Clean up handling of errors from outbound requests
 2018-08-02 17:38:24 +01:00
Richard van der Hoff
704c3e6239 Merge branch 'master' into develop 2018-08-02 15:43:30 +01:00
Richard van der Hoff
0bf5ec0db7 Check room visibility for /event/ requests 
Make sure that the user has permission to view the requeseted event for
/event/{eventId} and /room/{roomId}/event/{eventId} requests.

Also check that the event is in the given room for
/room/{roomId}/event/{eventId}, for sanity.
 2018-08-02 15:03:27 +01:00
Richard van der Hoff
b8d7d3996b
Merge pull request #3620 from fuzzmz/return-404-room-not-found 
return 404 if room not found
 2018-08-01 16:34:32 +01:00
Richard van der Hoff
018d75a148 Refactor code for turning HttpResponseException into SynapseError 
This commit replaces SynapseError.from_http_response_exception with
HttpResponseException.to_synapse_error.

The new method actually returns a ProxiedRequestError, which allows us to pass
through additional metadata from the API call.
 2018-08-01 16:02:46 +01:00
Amber Brown
da7785147d
Python 3: Convert some unicode/bytes uses (#3569) 2018-08-02 00:54:06 +10:00
Serban Constantin
70af98e361
return NotFoundError if room not found 
Per the Client-Server API[0] we should return
`M_NOT_FOUND` if the room isn't found instead
of generic SynapseError.

This ensures that /directory/list API returns
404 for room not found instead of 400.

[0]: https://matrix.org/docs/spec/client_server/unstable.html#get-matrix-client-r0-directory-list-room-roomid

Signed-off-by: Serban Constantin <serban.constantin@gmail.com>
 2018-07-31 21:47:23 +03:00
Matthew Hodgson
e9b2d047f6
make /context lazyload & filter aware (#3567) 
make /context lazyload & filter aware.
 2018-07-27 15:12:50 +01:00
Richard van der Hoff
03751a6420 Fix some looping_call calls which were broken in #3604 
It turns out that looping_call does check the deferred returned by its
callback, and (at least in the case of client_ips), we were relying on this,
and I broke it in #3604.

Update run_as_background_process to return the deferred, and make sure we
return it to clock.looping_call.
 2018-07-26 11:48:08 +01:00
Richard van der Hoff
371da42ae4 Wrap a number of things that run in the background 
This will reduce the number of "Starting db connection from sentinel context"
warnings, and will help with our metrics.
 2018-07-25 09:41:12 +01:00
Erik Johnston
0b0b24cb82 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/client_apis_move 2018-07-23 13:21:15 +01:00
Amber Brown
e1a237eaab
Admin API for creating new users (#3415) 2018-07-20 22:41:13 +10:00
Amber Brown
a97c845271
Move v1-only APIs into their own module & isolate deprecated ones (#3460) 2018-07-19 20:03:33 +10:00
Erik Johnston
bacdf0cbf9 Move RoomContextHandler out of Handlers 
This is in preparation for moving GET /context/ to a worker
 2018-07-18 15:33:03 +01:00
Erik Johnston
8cb8df55e9 Split MessageHandler into read only and writers 
This will let us call the read only parts from workers, and so be able
to move some APIs off of master, e.g. the `/state` API.
 2018-07-18 15:33:03 +01:00
Amber Brown
bc006b3c9d
Refactor REST API tests to use explicit reactors (#3351) 2018-07-17 20:43:18 +10:00
Krombel
78a9ddcf9a rerun isort with latest version 2018-07-16 14:23:25 +02:00
Krombel
4a27000548 check isort by travis 2018-07-16 13:57:33 +02:00
Amber Brown
8a4f05fefb
Fix develop because I broke it :( (#3535) 2018-07-14 09:51:00 +10:00
Amber Brown
8532953c04
Merge pull request #3534 from krombel/use_parse_and_asserts_from_servlet 
Use parse and asserts from http.servlet
 2018-07-14 09:09:19 +10:00
Amber Brown
a2374b2c7f
fix sytests 2018-07-14 07:52:58 +10:00
Amber Brown
33b60c01b5
Make auth & transactions more testable (#3499) 2018-07-14 07:34:49 +10:00
Krombel
516f960ad8 add changelog 2018-07-13 22:19:19 +02:00
Krombel
3366b9c534 rename assert_params_in_request to assert_params_in_dict 
the method "assert_params_in_request" does handle dicts and not
requests. A request body has to be parsed to json before this method
can be used
 2018-07-13 21:53:01 +02:00
Krombel
32fd6910d0 Use parse_{int,str} and assert from http.servlet 
parse_integer and parse_string can take a request and raise errors
in case we have wrong or missing params.
This PR tries to use them more to deduplicate some code and make it
better readable
 2018-07-13 21:40:14 +02:00
Richard van der Hoff
482d17b58b Merge branch 'develop' into rav/enforce_report_api 2018-07-12 09:56:28 +01:00
Amber Brown
49af402019 run isort 2018-07-09 16:09:20 +10:00
Amber Brown
6350bf925e
Attempt to be more performant on PyPy (#3462) 2018-06-28 14:49:57 +01:00
Matthew Hodgson
9570aa82eb update doc for deactivate API 2018-06-26 10:42:50 +01:00
Matthew Hodgson
1e788db430 add GDPR erase param to deactivate API 2018-06-26 10:26:54 +01:00
Erik Johnston
244484bf3c Revert "Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility"" 
This reverts commit 1d009013b3.
 2018-06-25 13:42:55 +01:00
Richard van der Hoff
1d009013b3 Revert "Merge pull request #3431 from matrix-org/rav/erasure_visibility" 
This reverts commit ce0d911156, reversing
changes made to b4a5d767a9.
 2018-06-22 16:35:10 +01:00
Erik Johnston
ce0d911156
Merge pull request #3431 from matrix-org/rav/erasure_visibility 
Support hiding events from deleted users
 2018-06-22 15:06:44 +01:00
Amber Brown
77ac14b960
Pass around the reactor explicitly (#3385) 2018-06-22 09:37:10 +01:00
Amber Brown
a61738b316
Remove run_on_reactor (#3395) 2018-06-14 18:27:37 +10:00
Richard van der Hoff
f1023ebf4b mark accounts as erased when requested 2018-06-12 09:53:18 +01:00
David Baker
187a546bff
Merge pull request #3276 from matrix-org/dbkr/unbind 
Remove email addresses / phone numbers from ID servers when they're removed from synapse
 2018-06-11 16:02:00 +01:00
Richard van der Hoff
f4caf3f83d fix log 2018-06-07 00:26:38 +01:00
Richard van der Hoff
0546715c18 Fix event-purge-by-ts admin API 
This got completely broken in 0.30.

Fixes #3300.
 2018-06-07 00:15:49 +01:00
Richard van der Hoff
7e15410f02 Enforce the specified API for report_event 
as per
https://matrix.org/docs/spec/client_server/unstable.html#post-matrix-client-r0-rooms-roomid-report-eventid
 2018-05-31 18:17:11 +01:00
Amber Brown
c936a52a9e
Consistently use six's iteritems and wrap lazy keys/values in list() if they're not meant to be lazy (#3307) 2018-05-31 19:03:47 +10:00
Amber Brown
1f69693347
Merge pull request #3244 from NotAFile/py3-six-4 
replace some iteritems with six
 2018-05-24 13:04:07 -05:00
Amber Brown
36501068d8
Merge pull request #3247 from NotAFile/py3-misc 
Misc Python3 fixes
 2018-05-24 12:58:37 -05:00
David Baker
77a23e2e05 Merge remote-tracking branch 'origin/develop' into dbkr/unbind 2018-05-24 16:20:53 +01:00
Erik Johnston
46345187cc
Merge pull request #3243 from NotAFile/py3-six-3 
Replace some more comparisons with six
 2018-05-24 16:08:57 +01:00
David Baker
9700d15611 pep8 2018-05-24 11:23:15 +01:00
David Baker
b3bff53178 Unbind 3pids when they're deleted too 2018-05-24 11:08:05 +01:00
Richard van der Hoff
7b36d06a69 Add a 'has_consented' template var to consent forms 
fixes #3260
 2018-05-22 14:58:34 +01:00
Richard van der Hoff
669400e22f Enable auto-escaping for the consent templates 
... to reduce the risk of somebody introducing an html injection attack...
 2018-05-22 14:58:34 +01:00
Richard van der Hoff
8810685df9 Stub out ServerNoticesSender on the workers 
... and have the sync endpoints call it directly rather than obsure indirection
via PresenceHandler
 2018-05-22 11:54:51 +01:00
Richard van der Hoff
d5dca9a04f Move consent config parsing into ConsentConfig 
turns out we need to reuse this, so it's better in the config class.
 2018-05-22 11:54:51 +01:00
Richard van der Hoff
d14d7b8fdc Rename 'version' param on user consent config 
we're going to use it for the version we require too.
 2018-05-22 11:54:51 +01:00
Richard van der Hoff
6e1cb54a05 Fix logcontext leak in HttpTransactionCache 
ONE DAY I WILL PURGE THE WORLD OF THIS EVIL
 2018-05-21 16:58:20 +01:00
Richard van der Hoff
6d6e7288fe Stop the transaction cache caching failures 
The transaction cache has some code which tries to stop it caching failures,
but if the callback function failed straight away, then things would happen
backwards and we'd end up with the failure stuck in the cache.
 2018-05-21 16:49:59 +01:00
Adrian Tschira
933bf2dd35 replace some iteritems with six 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:59:26 +02:00
Adrian Tschira
d9fe2b2d9d Replace some more comparisons with six 
plus a bonus b"" string I missed last time

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-05-19 17:56:31 +02:00
Adrian Tschira
aafb0f6b0d py3-ize url preview 2018-05-19 17:35:20 +02:00
Erik Johnston
fa30ac38cc
Merge pull request #3221 from matrix-org/erikj/purge_token 
Make purge_history operate on tokens
 2018-05-18 10:35:23 +01:00
Richard van der Hoff
c46367d0d7 Move RoomCreationHandler out of synapse.handlers.Handlers 
Handlers is deprecated nowadays, so let's move this out before I add a new
dependency on it.

Also fix the docstrings on create_room.
 2018-05-17 09:08:42 +01:00
Richard van der Hoff
8030a825c8
Merge pull request #3213 from matrix-org/rav/consent_handler 
ConsentResource to gather policy consent from users
 2018-05-16 07:19:18 +01:00
Erik Johnston
5f27ed75ad Make purge_history operate on tokens 
As we're soon going to change how topological_ordering works
 2018-05-15 16:23:50 +01:00
Richard van der Hoff
47815edcfa ConsentResource to gather policy consent from users 
Hopefully there are enough comments and docs in this that it makes sense on its
own.
 2018-05-15 15:11:59 +01:00
Richard van der Hoff
318711e139 Set Server header in SynapseRequest 
(instead of everywhere that writes a response. Or rather, the subset of places
which write responses where we haven't forgotten it).

This also means that we don't have to have the mysterious version_string
attribute in anything with a request handler.

Unfortunately it does mean that we have to pass the version string wherever we
instantiate a SynapseSite, which has been c&ped 150 times, but that is code
that ought to be cleaned up anyway really.
 2018-05-10 18:50:27 +01:00
Richard van der Hoff
645cb4bf06 Remove redundant request_handler decorator 
This is needless complexity; we might as well use the wrapper directly.

Also rename wrap_request_handler->wrap_json_request_handler.
 2018-05-10 12:19:53 +01:00
Konstantinos Sideris
88868b2839 notifications: Convert next_token to string according to the spec 
Currently the parameter is serialized as an integer.

Signed-off-by: Konstantinos Sideris <sideris.konstantin@gmail.com>
 2018-05-05 12:55:02 +03:00
Richard van der Hoff
902673e356
Merge pull request #3161 from NotAFile/remove-v1auth 
Make Client-Server API return 403 for invalid token
 2018-05-03 10:10:57 +01:00
Richard van der Hoff
be31adb036 Fix logcontext leak in media repo 
Make FileResponder.write_to_consumer uphold the logcontext contract
 2018-05-02 16:14:50 +01:00
Adrian Tschira
6495dbb326 Burminate v1auth 
This closes #2602

v1auth was created to account for the differences in status code between
the v1 and v2_alpha revisions of the protocol (401 vs 403 for invalid
tokens). However since those protocols were merged, this makes the r0
version/endpoint internally inconsistent, and violates the
specification for the r0 endpoint.

This might break clients that rely on this inconsistency with the
specification. This is said to affect the legacy angular reference
client. However, I feel that restoring parity with the spec is more
important. Either way, it is critical to inform developers about this
change, in case they rely on the illegal behaviour.

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-30 22:20:43 +02:00
Krombel
576b71dd3d add guard for None on purge_history api 2018-04-30 14:29:48 +02:00
Richard van der Hoff
dbf6f28d64
Merge pull request #3155 from NotAFile/py3-bytes-1 
more bytes strings
 2018-04-30 00:38:21 +01:00
Richard van der Hoff
aab2e4da60
Merge pull request #3140 from matrix-org/rav/use_run_in_background 
Use run_in_background in preference to preserve_fn
 2018-04-30 00:34:28 +01:00
Richard van der Hoff
1315d374cc
Merge pull request #3156 from NotAFile/py3-hmac-bytes 
Construct HMAC as bytes on py3
 2018-04-30 00:33:20 +01:00
Richard van der Hoff
9e2601f830
Merge pull request #3108 from NotAFile/py3-six-urlparse 
Use six.moves.urlparse
 2018-04-30 00:33:05 +01:00
Adrian Tschira
122593265b Construct HMAC as bytes on py3 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-29 00:19:41 +02:00
Adrian Tschira
e9143b6593 more bytes strings 
Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-29 00:13:57 +02:00
Richard van der Hoff
fc149b4eeb Merge remote-tracking branch 'origin/develop' into rav/use_run_in_background 2018-04-27 14:31:23 +01:00
Richard van der Hoff
2a13af23bc Use run_in_background in preference to preserve_fn 
While I was going through uses of preserve_fn for other PRs, I converted places
which only use the wrapped function once to use run_in_background, to avoid
creating the function object.
 2018-04-27 12:55:51 +01:00
Richard van der Hoff
9255a6cb17 Improve exception handling for background processes 
There were a bunch of places where we fire off a process to happen in the
background, but don't have any exception handling on it - instead relying on
the unhandled error being logged when the relevent deferred gets
garbage-collected.

This is unsatisfactory for a number of reasons:
 - logging on garbage collection is best-effort and may happen some time after
   the error, if at all
 - it can be hard to figure out where the error actually happened.
 - it is logged as a scary CRITICAL error which (a) I always forget to grep for
   and (b) it's not really CRITICAL if a background process we don't care about
   fails.

So this is an attempt to add exception handling to everything we fire off into
the background.
 2018-04-27 11:07:40 +01:00
Adrian Tschira
2a3c33ff03 Use six.moves.urlparse 
The imports were shuffled around a bunch in py3

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-15 21:22:43 +02:00
Erik Johnston
eaa2ebf20b
Merge pull request #3079 from matrix-org/erikj/limit_concurrent_sends 
Limit concurrent event sends for a room
 2018-04-10 16:43:58 +01:00
Richard van der Hoff
a3599dda97
Merge pull request #2996 from krombel/allow_auto_join_rooms 
move handling of auto_join_rooms to RegisterHandler
 2018-04-10 01:11:00 +01:00
Richard van der Hoff
87478c5a60
Merge pull request #3061 from NotAFile/add-some-byte-strings 
Add b prefixes to some strings that are bytes in py3
 2018-04-09 23:54:05 +01:00
Richard van der Hoff
c508b2f2f0
Merge pull request #3073 from NotAFile/use-six-reraise 
Replace old-style raise with six.reraise
 2018-04-09 23:53:40 +01:00
Richard van der Hoff
37354b55c9
Merge pull request #2938 from dklug/develop 
Return 401 for invalid access_token on logout
 2018-04-09 23:52:56 +01:00
Erik Johnston
56b0589865 Use create_and_send_nonmember_event everywhere 2018-04-09 12:04:18 +01:00
Adrian Tschira
4f40d058cc Replace old-style raise with six.reraise 
The old style raise is invalid syntax in python3. As noted in the docs,
this adds one more frame in the traceback, but I think this is
acceptable:

    <ipython-input-7-bcc5cba3de3f> in <module>()
         16     except:
         17         pass
    ---> 18     six.reraise(*x)

    /usr/lib/python3.6/site-packages/six.py in reraise(tp, value, tb)
        691             if value.__traceback__ is not tb:
        692                 raise value.with_traceback(tb)
    --> 693             raise value
        694         finally:
        695             value = None

    <ipython-input-7-bcc5cba3de3f> in <module>()
          9
         10 try:
    ---> 11     x()
         12 except:
         13     x = sys.exc_info()

Also note that this uses six, which is not formally a dependency yet,
but is included indirectly since most packages depend on it.

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-06 23:06:24 +02:00
Richard van der Hoff
c7f0969731
Merge pull request #2986 from jplatte/join_reponse_room_id 
Add room_id to the response of `rooms/{roomId}/join`
 2018-04-05 17:29:06 +01:00
Luke Barnard
104c0bc1d5 Use "/settings/" (plural) 2018-04-05 14:07:16 +01:00
Adrian Tschira
6168351877 Add b prefixes to some strings that are bytes in py3 
This has no effect on python2

Signed-off-by: Adrian Tschira <nota@notafile.com>
 2018-04-04 13:48:51 +02:00
Luke Barnard
eb8d8d6f57 Use join_policy API instead of joinable 
The API is now under
 /groups/$group_id/setting/m.join_policy

and expects a JSON blob of the shape

```json
{
  "m.join_policy": {
    "type": "invite"
  }
}
```

where "invite" could alternatively be "open".
 2018-04-03 16:16:40 +01:00
David Baker
c5de6987c2 This should probably be a PUT 2018-03-28 16:44:11 +01:00
David Baker
79452edeee Add joinability for groups 
Adds API to set the 'joinable' flag, and corresponding flag in the
table.
 2018-03-28 14:03:37 +01:00
Krombel
6152e253d8 Merge branch 'develop' of into allow_auto_join_rooms 2018-03-28 14:45:28 +02:00
Erik Johnston
fa72803490 Merge branch 'master' of github.com:matrix-org/synapse into develop 2018-03-19 11:41:01 +00:00
Erik Johnston
a8ce159be4 Replace some ujson with simplejson to make it work 2018-03-16 00:27:09 +00:00
Erik Johnston
926ba76e23 Replace ujson with simplejson 2018-03-15 23:43:31 +00:00
Krombel
91ea0202e6 move handling of auto_join_rooms to RegisterHandler 
Currently the handling of auto_join_rooms only works when a user
registers itself via public register api. Registrations via
registration_shared_secret and ModuleApi do not work

This auto_joins the users in the registration handler which enables
the auto join feature for all 3 registration paths.

This is related to issue #2725

Signed-Off-by: Matthias Kesler <krombel@krombel.de>
 2018-03-14 16:45:37 +01:00
Jonas Platte
47ce527f45 Add room_id to the response of rooms/{roomId}/join 
Fixes #2349
 2018-03-13 14:48:12 +01:00
Erik Johnston
f5160d4a3e RoomMembershipRestServlet doesn't handle /forget 
Due to the order we register the REST handlers `/forget` was handled by
the correct handler.
 2018-03-13 12:12:55 +00:00
Richard van der Hoff
e48c7aac4d Add transactional API to history purge 
Make the purge request return quickly, and allow scripts to poll for updates.
 2018-03-12 16:22:55 +00:00
Richard van der Hoff
f8bfcd7e0d Provide a means to pass a timestamp to purge_history 2018-03-05 14:37:23 +00:00
dklug
af7ed8e1ef Return 401 for invalid access_token on logout 
Signed-off-by: Duncan Klug <dklug@ucmerced.edu>
 2018-03-02 22:01:27 -08:00
Erik Johnston
784f036306 Move RoomMemberHandler out of Handlers 2018-03-01 14:36:50 +00:00
Erik Johnston
92c52df702 Make store_file use store_into_file 2018-02-14 17:55:18 +00:00
Erik Johnston
c0c9327fe0
Merge pull request #2854 from matrix-org/erikj/event_create_worker 
Create a worker for event creation
 2018-02-13 18:07:10 +00:00
Richard van der Hoff
8fd1a32456 Fix typos in purge api & doc 
* It's supposed to be purge_local_events, not ..._history
* Fix the doc to have valid json
 2018-02-13 13:09:39 +00:00
Erik Johnston
1026690cd2
Merge pull request #2857 from matrix-org/erikj/upload_store 
Tell storage providers about new file so they can upload
 2018-02-12 13:52:58 +00:00
Richard van der Hoff
10b34dbb9a
Merge pull request #2858 from matrix-org/rav/purge_updates 
delete_local_events for purge_room_history
 2018-02-09 14:11:00 +00:00
Richard van der Hoff
74fcbf741b delete_local_events for purge_history 
Add a flag which makes the purger delete local events
 2018-02-09 13:07:41 +00:00
Erik Johnston
5fa571a91b Tell storage providers about new file so they can upload 2018-02-07 13:35:08 +00:00
Erik Johnston
8ec2e638be Add event_creator worker 2018-02-07 10:32:32 +00:00
Erik Johnston
3e1e69ccaf Update copyright 2018-02-06 16:40:38 +00:00
Erik Johnston
3fa362502c Update places where we create events 2018-02-05 16:01:48 +00:00
Erik Johnston
1f881e0746
Merge pull request #2791 from matrix-org/erikj/media_storage_refactor 
Ensure media is in local cache before thumbnailing
 2018-02-05 11:28:52 +00:00
Richard van der Hoff
9c9356512e
Merge pull request #2845 from matrix-org/rav/urlcache_error_handling 
Handle url_previews with no content-type
 2018-02-02 15:27:52 +01:00
Travis Ralston
6e87b34f7b
Merge branch 'develop' into travis/admin-list-media 2018-02-01 18:05:47 -07:00
Richard van der Hoff
d5352cbba8 Handle url_previews with no content-type 
avoid failing with an exception if the remote server doesn't give us a
Content-Type header.

Also, clean up the exception handling a bit.
 2018-02-02 00:53:46 +00:00
Matthew Hodgson
9a72b70630 fix thinko on 3pid whitelisting 2018-01-24 11:07:47 +01:00
Matthew Hodgson
d32385336f
add ?ts massaging for ASes (#2754) 
blindly implement ?ts for AS. untested
 2018-01-23 09:59:06 +01:00
Matthew Hodgson
ab9f844aaf
Add federation_domain_whitelist option (#2820) 
Add federation_domain_whitelist

gives a way to restrict which domains your HS is allowed to federate with.
useful mainly for gracefully preventing a private but internet-connected HS from trying to federate to the wider public Matrix network
 2018-01-22 19:11:18 +01:00
Matthew Hodgson
d84f65255e
Merge pull request #2813 from matrix-org/matthew/registrations_require_3pid 
add registrations_require_3pid and allow_local_3pids
 2018-01-22 13:57:22 +00:00
Travis Ralston
5552ed9a7f Add an admin route to get all the media in a room 
This is intended to be used by administrators to monitor the media that is passing through their server, if they wish.

Signed-off-by: Travis Ralston <travpc@gmail.com>
 2018-01-20 22:37:53 -07:00
Matthew Hodgson
49fce04624 fix typo (thanks sytest) 2018-01-19 19:55:38 +00:00
Richard van der Hoff
b0d9e633ee
Merge pull request #2814 from matrix-org/rav/fix_urlcache_thumbs 
Use the right path for url_preview thumbnails
 2018-01-19 18:57:15 +00:00
Richard van der Hoff
ad7ec63d08 Use the right path for url_preview thumbnails 
This was introduced by #2627: we were overwriting the original media for url
previews with the thumbnails :/

(fixes https://github.com/vector-im/riot-web/issues/6012, hopefully)
 2018-01-19 18:29:39 +00:00
Matthew Hodgson
62d7d66ae5 oops, check all login types 2018-01-19 18:23:56 +00:00
Matthew Hodgson
293380bef7 trailing commas 2018-01-19 15:38:53 +00:00
Matthew Hodgson
447f4f0d5f rewrite based on PR feedback: 
* [ ] split config options into allowed_local_3pids and registrations_require_3pid
 * [ ] simplify and comment logic for picking registration flows
 * [ ] fix docstring and move check_3pid_allowed into a new util module
 * [ ] use check_3pid_allowed everywhere

@erikjohnston PTAL
 2018-01-19 15:33:55 +00:00
Matthew Hodgson
9d332e0f79 fix up v1, and improve errors 2018-01-19 00:53:58 +00:00
Matthew Hodgson
0af58f14ee fix pep8 2018-01-19 00:33:51 +00:00
Matthew Hodgson
28a6ccb49c add registrations_require_3pid 
lets homeservers specify a whitelist for 3PIDs that users are allowed to associate with.
Typically useful for stopping people from registering with non-work emails
 2018-01-19 00:19:58 +00:00
Erik Johnston
cd871a3057 Fix storage provider bug introduced when renamed to store_local 2018-01-18 18:37:59 +00:00
Erik Johnston
8ff6726c0d
Merge pull request #2812 from matrix-org/erikj/media_storage_provider_config 
Make storage providers configurable
 2018-01-18 18:33:57 +00:00
Erik Johnston
3fe2bae857 Missing staticmethod 2018-01-18 17:11:45 +00:00
Erik Johnston
aae77da73f Fixup comments 2018-01-18 17:11:29 +00:00
Erik Johnston
9a89dae8c5 Fix typo in thumbnail resource causing access times to be incorrect 2018-01-18 15:06:24 +00:00
Erik Johnston
0af5dc63a8 Make storage providers more configurable 2018-01-18 14:07:21 +00:00
Erik Johnston
2cf6a7bc20 Use better file consumer 2018-01-18 12:00:46 +00:00
Erik Johnston
4a53f3a3e8 Ensure media is in local cache before thumbnailing 2018-01-18 12:00:46 +00:00
Erik Johnston
300edc2348 Update last access time when thumbnails are viewed 2018-01-17 10:24:43 +00:00
Erik Johnston
05f98a2224 Keep track of last access time for local media 2018-01-17 10:24:43 +00:00
Erik Johnston
d728c47142 Add docstring 2018-01-17 10:06:14 +00:00
Erik Johnston
d863f68cab Use local vars 2018-01-16 16:24:15 +00:00
Erik Johnston
6368e5c0ab Change _generate_thumbnails to take media_type 2018-01-16 16:17:38 +00:00
Erik Johnston
0a90d9ede4 Move setting of file_id up to caller 2018-01-16 16:03:05 +00:00
Erik Johnston
5dfc83704b Fix typo 2018-01-16 14:32:56 +00:00
Erik Johnston
307f88dfb6 Fix up log lines 2018-01-16 13:53:52 +00:00
Erik Johnston
9795b9ebb1 Correctly use server_name/file_id when generating/fetching remote thumbnails 2018-01-16 12:02:06 +00:00
Erik Johnston
c5b589f2e8 Log when we respond with 404 2018-01-16 12:01:40 +00:00
Erik Johnston
a4c5e4a645 Fix thumbnailing remote files 2018-01-16 11:37:50 +00:00
Erik Johnston
1159abbdd2
Merge pull request #2767 from matrix-org/erikj/media_storage_refactor 
Refactor MediaRepository to separate out storage
 2018-01-16 10:23:50 +00:00
Richard van der Hoff
21bf87a146 Reinstate media download on thumbnail request 
We need to actually download the remote media when we get a request for a
thumbnail.
 2018-01-12 15:38:06 +00:00
Erik Johnston
694f1c1b18 Fix up comments 2018-01-12 15:02:46 +00:00
Erik Johnston
e21370ba54 Correctly reraise exception 2018-01-12 14:44:02 +00:00
Erik Johnston
85a4d78213 Make Responder a context manager 2018-01-12 13:32:03 +00:00
Erik Johnston
dcc8eded41 Add missing class var 2018-01-12 13:16:27 +00:00
Erik Johnston
81391fa162 Merge branch 'develop' of github.com:matrix-org/synapse into erikj/media_storage_refactor 2018-01-12 11:28:49 +00:00
Erik Johnston
1e4edd1717 Remove unnecessary condition 2018-01-12 11:28:32 +00:00
Erik Johnston
c6c009603c Remove unused variables 2018-01-12 11:24:05 +00:00
Erik Johnston
4d88958cf6 Make class var local 2018-01-12 11:23:54 +00:00
Erik Johnston
227c491510 Comments 2018-01-12 11:22:41 +00:00
Richard van der Hoff
5f23b6d5ea
Merge pull request #2766 from matrix-org/rav/room_event 
Add /room/{id}/event/{id} to synapse
 2018-01-11 14:47:18 +00:00
Erik Johnston
8f03aa9f61 Add StorageProvider concept 2018-01-09 16:16:12 +00:00
Erik Johnston
2442e9876c Make PreviewUrlResource use MediaStorage 2018-01-09 16:15:07 +00:00
Erik Johnston
9d30a7691c Make ThumbnailResource use MediaStorage 2018-01-09 16:15:07 +00:00
Erik Johnston
9e20840e02 Use MediaStorage for remote media 2018-01-09 16:15:07 +00:00
Erik Johnston
dd3092c3a3 Use MediaStorage for local files 2018-01-09 16:15:07 +00:00
Erik Johnston
ada470bccb Add MediaStorage class 2018-01-09 16:15:07 +00:00
Erik Johnston
1ee787912b Add some helper classes 2018-01-09 16:15:07 +00:00
Erik Johnston
47ca5eb882 Split out add_file_headers 2018-01-09 16:15:07 +00:00
Erik Johnston
b6c9deffda Remove dead TODO 2018-01-09 15:53:23 +00:00
Richard van der Hoff
51c9d9ed65 Add /room/{id}/event/{id} to synapse 
Turns out that there is a valid usecase for retrieving event by id (notably
having received a push), but event ids should be scoped to room, so /event/{id}
is wrong.
 2018-01-09 14:39:12 +00:00
Erik Johnston
b30cd5b107 Remove dead code related to default thumbnails 2018-01-09 14:38:33 +00:00
Richard van der Hoff
75c1b8df01 Better logging when login can't find a 3pid 2017-12-20 19:31:00 +00:00
Richard van der Hoff
16ec3805e5 Fix error when deleting devices 
This was introduced in d7ea8c4 / PR #2728
 2017-12-05 09:49:22 +00:00
Richard van der Hoff
d7ea8c4800 Factor out a validate_user_via_ui_auth method 
Collect together all the places that validate a logged-in user via UI auth.
 2017-12-05 09:42:30 +00:00
Richard van der Hoff
d5f9fb06b0 Refactor UI auth implementation 
Instead of returning False when auth is incomplete, throw an exception which
can be caught with a wrapper.
 2017-12-05 09:40:05 +00:00
Richard van der Hoff
ad7e570d07 Delete devices in various logout situations 
Make sure that we delete devices whenever a user is logged out due to any of
the following situations:

 * /logout
 * /logout_all
 * change password
 * deactivate account (by the user or by an admin)
 * invalidate access token from a dynamic module

Fixes #2672.
 2017-11-29 16:44:35 +00:00